2024-01-21T22:05:40.882 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-01-21T22:05:40.882 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/vshankar-2024-01-19_10:13:34-fs-wip-vshankar-testing-20240119.075157-testing-default-smithi/7522024 --verbose --name vshankar-2024-01-19_10:13:34-fs-wip-vshankar-testing-20240119.075157-testing-default-smithi --description fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/dbench validater/valgrind} --owner scheduled_vshankar@teuthology -- /home/teuthworker/archive/vshankar-2024-01-19_10:13:34-fs-wip-vshankar-testing-20240119.075157-testing-default-smithi/7522024/orig.config.yaml 2024-01-21T22:05:40.926 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-21T22:05:41.010 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-01-19_10:13:34-fs-wip-vshankar-testing-20240119.075157-testing-default-smithi/7522024 branch: wip-vshankar-testing-20240119.075157 description: fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/dbench validater/valgrind} email: vshankar@redhat.com first_in_suite: false job_id: '7522024' kernel: client: branch: testing flavor: default kdb: true sha1: 06652275eb934672217aaf9d4180f4a90395eb16 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2024-01-19_10:13:34-fs-wip-vshankar-testing-20240119.075157-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240119.075157 ceph: cephfs: session_timeout: 300 conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mds beacon grace: 60 mds heartbeat grace: 60 ms die on skipped message: false osd heartbeat grace: 60 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds valgrind exit: true mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon osd crush smoke test: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd fast shutdown: false osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow request - SLOW_OPS - MON_DOWN sha1: eb1b049f9003ac37e0a3ab33496ad1924d301254 valgrind: mds: - --tool=memcheck mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes watchdog: daemon_restart: normal ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: valgrind: - --tool=memcheck - --leak-check=full - --show-reachable=yes install: ceph: debuginfo: true flavor: default sha1: eb1b049f9003ac37e0a3ab33496ad1924d301254 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240119.075157 sha1: eb1b049f9003ac37e0a3ab33496ad1924d301254 owner: scheduled_vshankar@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 3392 sha1: eb1b049f9003ac37e0a3ab33496ad1924d301254 sleep_before_teardown: 0 subset: 163/1024 suite: fs suite_branch: wip-vshankar-testing-20240119.075157 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_eb1b049f9003ac37e0a3ab33496ad1924d301254/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: eb1b049f9003ac37e0a3ab33496ad1924d301254 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLyJGSaW2X8YY71fG8SngBVERcLbGclRlIadn2c0LipJRJCrI6qJ6FL8dD81NbSnvf5uuIDaFsJZ2adGvW7BFE4= smithi152.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEeNVGhw4BYJUYb1i7A07lKn6tOaA9Y9fy0bgq+8084xBMhXWcHRfdSuRm+R4HCb2qW/u8uwFcdu2WAnl5J4ySI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: cd45576300487d997e5a85abed65500b9f5d143b timestamp: 2024-01-19_10:13:34 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-01-21T22:05:41.011 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_eb1b049f9003ac37e0a3ab33496ad1924d301254/qa; will attempt to use it 2024-01-21T22:05:41.011 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_eb1b049f9003ac37e0a3ab33496ad1924d301254/qa/tasks 2024-01-21T22:05:41.012 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-01-21T22:05:41.013 INFO:teuthology.task.internal:Checking packages... 2024-01-21T22:05:41.037 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'eb1b049f9003ac37e0a3ab33496ad1924d301254' 2024-01-21T22:05:41.037 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-01-21T22:05:41.037 INFO:teuthology.packaging:ref: None 2024-01-21T22:05:41.037 INFO:teuthology.packaging:tag: None 2024-01-21T22:05:41.037 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240119.075157 2024-01-21T22:05:41.038 INFO:teuthology.packaging:sha1: eb1b049f9003ac37e0a3ab33496ad1924d301254 2024-01-21T22:05:41.038 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240119.075157 2024-01-21T22:05:41.186 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-811.geb1b049f 2024-01-21T22:05:41.187 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-01-21T22:05:41.196 INFO:teuthology.task.internal:no buildpackages task found 2024-01-21T22:05:41.196 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-01-21T22:05:41.218 INFO:teuthology.task.internal:Saving configuration 2024-01-21T22:05:41.236 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-01-21T22:05:41.242 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-21T22:05:41.266 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-19_10:13:34-fs-wip-vshankar-testing-20240119.075157-testing-default-smithi/7522024', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-21 22:00:19.973383', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLyJGSaW2X8YY71fG8SngBVERcLbGclRlIadn2c0LipJRJCrI6qJ6FL8dD81NbSnvf5uuIDaFsJZ2adGvW7BFE4='} 2024-01-21T22:05:41.289 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi152.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-19_10:13:34-fs-wip-vshankar-testing-20240119.075157-testing-default-smithi/7522024', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-21 22:00:19.975340', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEeNVGhw4BYJUYb1i7A07lKn6tOaA9Y9fy0bgq+8084xBMhXWcHRfdSuRm+R4HCb2qW/u8uwFcdu2WAnl5J4ySI='} 2024-01-21T22:05:41.289 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-01-21T22:05:41.295 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-01-21T22:05:41.295 INFO:teuthology.task.internal:roles: ubuntu@smithi152.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-01-21T22:05:41.295 INFO:teuthology.run_tasks:Running task console_log... 2024-01-21T22:05:41.371 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff1075db160>, signals=[15]) 2024-01-21T22:05:41.372 INFO:teuthology.run_tasks:Running task internal.connect... 2024-01-21T22:05:41.378 INFO:teuthology.task.internal:Opening connections... 2024-01-21T22:05:41.379 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-01-21T22:05:41.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-21T22:05:41.471 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi152.front.sepia.ceph.com 2024-01-21T22:05:41.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-21T22:05:41.561 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-01-21T22:05:41.569 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-01-21T22:05:41.586 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-01-21T22:05:41.587 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-01-21T22:05:41.643 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-01-21T22:05:41.643 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2024-01-21T22:05:41.643 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-01-21T22:05:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-01-21T22:05:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2024-01-21T22:05:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2024-01-21T22:05:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-21T22:05:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-01-21T22:05:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2024-01-21T22:05:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-21T22:05:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-01-21T22:05:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-21T22:05:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-21T22:05:41.644 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-21T22:05:41.645 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-01-21T22:05:41.666 DEBUG:teuthology.orchestra.run.smithi152:> uname -m 2024-01-21T22:05:41.683 INFO:teuthology.orchestra.run.smithi152.stdout:x86_64 2024-01-21T22:05:41.683 DEBUG:teuthology.orchestra.run.smithi152:> cat /etc/os-release 2024-01-21T22:05:41.739 INFO:teuthology.orchestra.run.smithi152.stdout:NAME="CentOS Stream" 2024-01-21T22:05:41.740 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION="9" 2024-01-21T22:05:41.740 INFO:teuthology.orchestra.run.smithi152.stdout:ID="centos" 2024-01-21T22:05:41.740 INFO:teuthology.orchestra.run.smithi152.stdout:ID_LIKE="rhel fedora" 2024-01-21T22:05:41.740 INFO:teuthology.orchestra.run.smithi152.stdout:VERSION_ID="9" 2024-01-21T22:05:41.740 INFO:teuthology.orchestra.run.smithi152.stdout:PLATFORM_ID="platform:el9" 2024-01-21T22:05:41.740 INFO:teuthology.orchestra.run.smithi152.stdout:PRETTY_NAME="CentOS Stream 9" 2024-01-21T22:05:41.740 INFO:teuthology.orchestra.run.smithi152.stdout:ANSI_COLOR="0;31" 2024-01-21T22:05:41.740 INFO:teuthology.orchestra.run.smithi152.stdout:LOGO="fedora-logo-icon" 2024-01-21T22:05:41.740 INFO:teuthology.orchestra.run.smithi152.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-01-21T22:05:41.740 INFO:teuthology.orchestra.run.smithi152.stdout:HOME_URL="https://centos.org/" 2024-01-21T22:05:41.740 INFO:teuthology.orchestra.run.smithi152.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-01-21T22:05:41.740 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-01-21T22:05:41.741 INFO:teuthology.orchestra.run.smithi152.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-01-21T22:05:41.743 INFO:teuthology.lock.ops:Updating smithi152.front.sepia.ceph.com on lock server 2024-01-21T22:05:41.766 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-01-21T22:05:41.776 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-01-21T22:05:41.781 INFO:teuthology.task.internal:Checking for old test directory... 2024-01-21T22:05:41.781 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-01-21T22:05:41.784 DEBUG:teuthology.orchestra.run.smithi152:> test '!' -e /home/ubuntu/cephtest 2024-01-21T22:05:41.800 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-01-21T22:05:41.806 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-01-21T22:05:41.806 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-01-21T22:05:41.840 DEBUG:teuthology.orchestra.run.smithi152:> test -z $(ls -A /var/lib/ceph) 2024-01-21T22:05:41.869 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-01-21T22:05:41.923 INFO:teuthology.run_tasks:Running task kernel... 2024-01-21T22:05:41.938 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '06652275eb934672217aaf9d4180f4a90395eb16'} 2024-01-21T22:05:41.938 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-01-21T22:05:41.938 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-01-21T22:05:41.938 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-21T22:05:41.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-21T22:05:41.955 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-01-21T22:05:42.253 DEBUG:teuthology.task.kernel:sha1 for client.0 is 06652275eb934672217aaf9d4180f4a90395eb16 2024-01-21T22:05:42.253 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "06652275eb934672217aaf9d4180f4a90395eb16"... 2024-01-21T22:05:42.253 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-01-21T22:05:42.270 INFO:teuthology.orchestra.run.smithi138.stdout:5.14.0-361.el9.x86_64 2024-01-21T22:05:42.270 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 06652275eb934672217aaf9d4180f4a90395eb16 2024-01-21T22:05:42.271 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-01-21T22:05:42.271 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum upgrade -y linux-firmware 2024-01-21T22:05:44.500 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS-9 - AppStream 14 MB/s | 6.4 MB 00:00 2024-01-21T22:05:45.828 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS-9 - BaseOS 7.4 MB/s | 2.0 MB 00:00 2024-01-21T22:05:47.266 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 7.2 MB/s | 8.0 MB 00:01 2024-01-21T22:05:50.125 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-01-21T22:05:54.434 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 7.3 MB/s | 5.9 MB 00:00 2024-01-21T22:05:55.777 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 15 kB 00:00 2024-01-21T22:05:56.712 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 25 MB/s | 20 MB 00:00 2024-01-21T22:06:01.720 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-01-21T22:06:04.030 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-01-21T22:06:04.040 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-01-21T22:06:04.040 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-01-21T22:06:04.041 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-01-21T22:06:04.041 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2024-01-21T22:06:04.041 INFO:teuthology.orchestra.run.smithi138.stdout: linux-firmware noarch 20240109-142.el9 CentOS-BaseOS 375 M 2024-01-21T22:06:04.041 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:06:04.041 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-01-21T22:06:04.041 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-01-21T22:06:04.041 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 1 Package 2024-01-21T22:06:04.041 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:06:04.041 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 375 M 2024-01-21T22:06:04.041 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-01-21T22:06:11.587 INFO:teuthology.orchestra.run.smithi138.stdout:linux-firmware-20240109-142.el9.noarch.rpm 50 MB/s | 375 MB 00:07 2024-01-21T22:06:11.588 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-01-21T22:06:11.588 INFO:teuthology.orchestra.run.smithi138.stdout:Total 50 MB/s | 375 MB 00:07 2024-01-21T22:06:11.589 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-01-21T22:06:11.672 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-01-21T22:06:11.672 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-01-21T22:06:14.090 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-01-21T22:06:14.090 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-01-21T22:06:20.544 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-01-21T22:06:20.799 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : linux-firmware-20240109-142.el9.noarch 1/2 2024-01-21T22:06:20.888 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2024-01-21T22:06:22.307 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2024-01-21T22:06:22.307 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : linux-firmware-20240109-142.el9.noarch 1/2 2024-01-21T22:06:22.717 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2024-01-21T22:06:22.717 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:06:22.717 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2024-01-21T22:06:22.718 INFO:teuthology.orchestra.run.smithi138.stdout: linux-firmware-20240109-142.el9.noarch 2024-01-21T22:06:22.718 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:06:22.718 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-01-21T22:06:23.017 INFO:teuthology.task.kernel:Downloading kernel 06652275eb934672217aaf9d4180f4a90395eb16 on client.0... 2024-01-21T22:06:23.018 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=06652275eb934672217aaf9d4180f4a90395eb16 2024-01-21T22:06:23.166 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/06652275eb934672217aaf9d4180f4a90395eb16/centos/9/flavors/default/x86_64/ 2024-01-21T22:06:23.166 DEBUG:teuthology.orchestra.run.smithi138:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.7.0_g06652275eb93-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://1.chacra.ceph.com/r/kernel/testing/06652275eb934672217aaf9d4180f4a90395eb16/centos/9/flavors/default/x86_64/ --input-file=- 2024-01-21T22:06:23.169 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi138.front.sepia.ceph.com to complete... 2024-01-21T22:06:34.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21 22:06:34 URL:https://1.chacra.ceph.com/r/kernel/testing/06652275eb934672217aaf9d4180f4a90395eb16/centos/9/flavors/default/x86_64/kernel-6.7.0_g06652275eb93-1.x86_64.rpm [330048165/330048165] -> "/tmp/kernel.x86_64.rpm" [1] 2024-01-21T22:06:34.827 INFO:teuthology.orchestra.run.smithi138.stderr:FINISHED --2024-01-21 22:06:34-- 2024-01-21T22:06:34.827 INFO:teuthology.orchestra.run.smithi138.stderr:Total wall clock time: 12s 2024-01-21T22:06:34.827 INFO:teuthology.orchestra.run.smithi138.stderr:Downloaded: 1 files, 315M in 11s (28.1 MB/s) 2024-01-21T22:06:34.829 INFO:teuthology.task.kernel:Installing kernel 06652275eb934672217aaf9d4180f4a90395eb16 on client.0... 2024-01-21T22:06:34.829 DEBUG:teuthology.orchestra.run.smithi138:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-01-21T22:06:36.725 INFO:teuthology.orchestra.run.smithi138.stdout:Verifying... ######################################## 2024-01-21T22:06:36.857 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing... ######################################## 2024-01-21T22:06:36.936 INFO:teuthology.orchestra.run.smithi138.stdout:Updating / installing... 2024-01-21T22:06:44.322 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-6.7.0_g06652275eb93-1 ######################################## 2024-01-21T22:07:01.295 INFO:teuthology.orchestra.run.smithi138.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-01-21T22:07:23.375 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi138.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-01-21T22:07:23.375 DEBUG:teuthology.orchestra.run.smithi138:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-01-21T22:07:23.450 INFO:teuthology.orchestra.run.smithi138.stdout:/boot/System.map-6.7.0-g06652275eb93 2024-01-21T22:07:23.451 INFO:teuthology.orchestra.run.smithi138.stdout:/boot/config-6.7.0-g06652275eb93 2024-01-21T22:07:23.451 INFO:teuthology.orchestra.run.smithi138.stdout:/boot/vmlinuz-6.7.0-g06652275eb93 2024-01-21T22:07:23.451 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93 2024-01-21T22:07:23.451 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel 2024-01-21T22:07:23.451 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch 2024-01-21T22:07:23.451 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86 2024-01-21T22:07:23.451 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/crypto 2024-01-21T22:07:23.451 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-01-21T22:07:23.451 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-01-21T22:07:23.451 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-01-21T22:07:23.451 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-01-21T22:07:23.451 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-01-21T22:07:23.451 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-01-21T22:07:23.452 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-01-21T22:07:23.452 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/crypto/crc32c-intel.ko 2024-01-21T22:07:23.452 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-01-21T22:07:23.452 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-01-21T22:07:23.452 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-01-21T22:07:23.452 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-01-21T22:07:23.452 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/crypto/serpent-avx2.ko 2024-01-21T22:07:23.452 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-01-21T22:07:23.452 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-01-21T22:07:23.452 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-01-21T22:07:23.452 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-01-21T22:07:23.452 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-01-21T22:07:23.452 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-01-21T22:07:23.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-01-21T22:07:23.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/kernel 2024-01-21T22:07:23.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/kernel/cpu 2024-01-21T22:07:23.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/kernel/cpu/mce 2024-01-21T22:07:23.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-01-21T22:07:23.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/kvm 2024-01-21T22:07:23.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/kvm/kvm-amd.ko 2024-01-21T22:07:23.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/kvm/kvm-intel.ko 2024-01-21T22:07:23.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/arch/x86/kvm/kvm.ko 2024-01-21T22:07:23.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto 2024-01-21T22:07:23.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/ansi_cprng.ko 2024-01-21T22:07:23.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/anubis.ko 2024-01-21T22:07:23.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/arc4.ko 2024-01-21T22:07:23.453 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/async_tx 2024-01-21T22:07:23.454 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/async_tx/async_memcpy.ko 2024-01-21T22:07:23.454 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/async_tx/async_pq.ko 2024-01-21T22:07:23.454 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/async_tx/async_raid6_recov.ko 2024-01-21T22:07:23.454 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/async_tx/async_tx.ko 2024-01-21T22:07:23.454 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/async_tx/async_xor.ko 2024-01-21T22:07:23.454 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/async_tx/raid6test.ko 2024-01-21T22:07:23.454 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/authenc.ko 2024-01-21T22:07:23.454 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/authencesn.ko 2024-01-21T22:07:23.454 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/blake2b_generic.ko 2024-01-21T22:07:23.454 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/blowfish_common.ko 2024-01-21T22:07:23.454 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/blowfish_generic.ko 2024-01-21T22:07:23.454 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/camellia_generic.ko 2024-01-21T22:07:23.455 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/cast5_generic.ko 2024-01-21T22:07:23.455 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/cast6_generic.ko 2024-01-21T22:07:23.455 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/cast_common.ko 2024-01-21T22:07:23.455 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/ccm.ko 2024-01-21T22:07:23.455 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/cmac.ko 2024-01-21T22:07:23.455 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/crc32_generic.ko 2024-01-21T22:07:23.455 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/crypto_engine.ko 2024-01-21T22:07:23.455 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/crypto_user.ko 2024-01-21T22:07:23.455 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/cts.ko 2024-01-21T22:07:23.455 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/des_generic.ko 2024-01-21T22:07:23.455 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/dh_generic.ko 2024-01-21T22:07:23.455 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/echainiv.ko 2024-01-21T22:07:23.455 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/essiv.ko 2024-01-21T22:07:23.455 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/fcrypt.ko 2024-01-21T22:07:23.456 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/khazad.ko 2024-01-21T22:07:23.456 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/lz4.ko 2024-01-21T22:07:23.456 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/lz4hc.ko 2024-01-21T22:07:23.456 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/md4.ko 2024-01-21T22:07:23.456 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/michael_mic.ko 2024-01-21T22:07:23.456 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/pcbc.ko 2024-01-21T22:07:23.456 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/pcrypt.ko 2024-01-21T22:07:23.456 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/rmd160.ko 2024-01-21T22:07:23.456 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/seed.ko 2024-01-21T22:07:23.456 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/serpent_generic.ko 2024-01-21T22:07:23.456 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/tcrypt.ko 2024-01-21T22:07:23.456 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/tea.ko 2024-01-21T22:07:23.456 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/twofish_common.ko 2024-01-21T22:07:23.457 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/twofish_generic.ko 2024-01-21T22:07:23.457 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/vmac.ko 2024-01-21T22:07:23.457 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/wp512.ko 2024-01-21T22:07:23.457 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/xcbc.ko 2024-01-21T22:07:23.457 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/xor.ko 2024-01-21T22:07:23.457 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/crypto/xxhash_generic.ko 2024-01-21T22:07:23.457 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers 2024-01-21T22:07:23.457 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/acpi 2024-01-21T22:07:23.457 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/acpi/acpi_ipmi.ko 2024-01-21T22:07:23.457 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/acpi/acpi_pad.ko 2024-01-21T22:07:23.457 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/acpi/custom_method.ko 2024-01-21T22:07:23.457 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/acpi/ec_sys.ko 2024-01-21T22:07:23.457 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/acpi/platform_profile.ko 2024-01-21T22:07:23.457 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/acpi/sbs.ko 2024-01-21T22:07:23.458 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/acpi/sbshc.ko 2024-01-21T22:07:23.458 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/acpi/video.ko 2024-01-21T22:07:23.458 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata 2024-01-21T22:07:23.458 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/acard-ahci.ko 2024-01-21T22:07:23.458 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/ahci_platform.ko 2024-01-21T22:07:23.458 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/ata_generic.ko 2024-01-21T22:07:23.458 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/libahci_platform.ko 2024-01-21T22:07:23.458 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_acpi.ko 2024-01-21T22:07:23.458 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_ali.ko 2024-01-21T22:07:23.458 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_amd.ko 2024-01-21T22:07:23.458 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_artop.ko 2024-01-21T22:07:23.458 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_atiixp.ko 2024-01-21T22:07:23.458 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_atp867x.ko 2024-01-21T22:07:23.458 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_cmd640.ko 2024-01-21T22:07:23.458 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_cmd64x.ko 2024-01-21T22:07:23.459 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_cypress.ko 2024-01-21T22:07:23.459 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_efar.ko 2024-01-21T22:07:23.459 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_hpt366.ko 2024-01-21T22:07:23.459 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_hpt37x.ko 2024-01-21T22:07:23.459 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_hpt3x2n.ko 2024-01-21T22:07:23.459 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_hpt3x3.ko 2024-01-21T22:07:23.459 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_it8213.ko 2024-01-21T22:07:23.459 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_it821x.ko 2024-01-21T22:07:23.459 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_jmicron.ko 2024-01-21T22:07:23.459 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_marvell.ko 2024-01-21T22:07:23.459 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_mpiix.ko 2024-01-21T22:07:23.459 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_netcell.ko 2024-01-21T22:07:23.459 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_ninja32.ko 2024-01-21T22:07:23.459 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_ns87410.ko 2024-01-21T22:07:23.460 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_ns87415.ko 2024-01-21T22:07:23.460 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_oldpiix.ko 2024-01-21T22:07:23.460 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_opti.ko 2024-01-21T22:07:23.460 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_optidma.ko 2024-01-21T22:07:23.460 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_pcmcia.ko 2024-01-21T22:07:23.460 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_pdc2027x.ko 2024-01-21T22:07:23.460 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-01-21T22:07:23.460 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_piccolo.ko 2024-01-21T22:07:23.460 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_rdc.ko 2024-01-21T22:07:23.460 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_sch.ko 2024-01-21T22:07:23.460 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_serverworks.ko 2024-01-21T22:07:23.460 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_sil680.ko 2024-01-21T22:07:23.460 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_sis.ko 2024-01-21T22:07:23.460 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_sl82c105.ko 2024-01-21T22:07:23.461 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_triflex.ko 2024-01-21T22:07:23.461 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pata_via.ko 2024-01-21T22:07:23.461 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/pdc_adma.ko 2024-01-21T22:07:23.461 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/sata_inic162x.ko 2024-01-21T22:07:23.461 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/sata_mv.ko 2024-01-21T22:07:23.461 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/sata_nv.ko 2024-01-21T22:07:23.461 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/sata_promise.ko 2024-01-21T22:07:23.461 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/sata_qstor.ko 2024-01-21T22:07:23.461 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/sata_sil.ko 2024-01-21T22:07:23.461 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/sata_sil24.ko 2024-01-21T22:07:23.461 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/sata_sis.ko 2024-01-21T22:07:23.461 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/sata_svw.ko 2024-01-21T22:07:23.461 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/sata_sx4.ko 2024-01-21T22:07:23.461 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/sata_uli.ko 2024-01-21T22:07:23.461 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/sata_via.ko 2024-01-21T22:07:23.462 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ata/sata_vsc.ko 2024-01-21T22:07:23.462 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/auxdisplay 2024-01-21T22:07:23.462 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/auxdisplay/ks0108.ko 2024-01-21T22:07:23.462 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/base 2024-01-21T22:07:23.462 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/base/regmap 2024-01-21T22:07:23.462 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/base/regmap/regmap-i2c.ko 2024-01-21T22:07:23.462 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/base/regmap/regmap-mmio.ko 2024-01-21T22:07:23.462 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/bcma 2024-01-21T22:07:23.462 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/bcma/bcma.ko 2024-01-21T22:07:23.462 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/block 2024-01-21T22:07:23.462 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/block/aoe 2024-01-21T22:07:23.462 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/block/aoe/aoe.ko 2024-01-21T22:07:23.462 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/block/brd.ko 2024-01-21T22:07:23.462 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/block/drbd 2024-01-21T22:07:23.463 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/block/drbd/drbd.ko 2024-01-21T22:07:23.463 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/block/floppy.ko 2024-01-21T22:07:23.463 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/block/loop.ko 2024-01-21T22:07:23.463 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/block/mtip32xx 2024-01-21T22:07:23.463 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-01-21T22:07:23.463 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/block/nbd.ko 2024-01-21T22:07:23.463 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/block/null_blk 2024-01-21T22:07:23.463 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/block/null_blk/null_blk.ko 2024-01-21T22:07:23.463 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/block/pktcdvd.ko 2024-01-21T22:07:23.463 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/block/rbd.ko 2024-01-21T22:07:23.463 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/block/virtio_blk.ko 2024-01-21T22:07:23.463 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/block/xen-blkback 2024-01-21T22:07:23.463 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-01-21T22:07:23.463 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/block/xen-blkfront.ko 2024-01-21T22:07:23.463 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/block/zram 2024-01-21T22:07:23.463 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/block/zram/zram.ko 2024-01-21T22:07:23.464 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char 2024-01-21T22:07:23.464 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/hangcheck-timer.ko 2024-01-21T22:07:23.464 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/hw_random 2024-01-21T22:07:23.464 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/hw_random/amd-rng.ko 2024-01-21T22:07:23.464 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/hw_random/intel-rng.ko 2024-01-21T22:07:23.464 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-01-21T22:07:23.464 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/hw_random/via-rng.ko 2024-01-21T22:07:23.464 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/hw_random/virtio-rng.ko 2024-01-21T22:07:23.464 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/ipmi 2024-01-21T22:07:23.464 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-01-21T22:07:23.464 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-01-21T22:07:23.464 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-01-21T22:07:23.464 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/ipmi/ipmi_si.ko 2024-01-21T22:07:23.464 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-01-21T22:07:23.464 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-01-21T22:07:23.465 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/lp.ko 2024-01-21T22:07:23.465 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/mwave 2024-01-21T22:07:23.465 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/mwave/mwave.ko 2024-01-21T22:07:23.465 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/ppdev.ko 2024-01-21T22:07:23.465 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/tlclk.ko 2024-01-21T22:07:23.465 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/tpm 2024-01-21T22:07:23.465 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/tpm/tpm.ko 2024-01-21T22:07:23.465 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/tpm/tpm_atmel.ko 2024-01-21T22:07:23.465 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/tpm/tpm_infineon.ko 2024-01-21T22:07:23.465 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/tpm/tpm_nsc.ko 2024-01-21T22:07:23.465 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/tpm/tpm_tis.ko 2024-01-21T22:07:23.465 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-01-21T22:07:23.465 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/uv_mmtimer.ko 2024-01-21T22:07:23.465 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/char/virtio_console.ko 2024-01-21T22:07:23.465 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/cpufreq 2024-01-21T22:07:23.466 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-01-21T22:07:23.466 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-01-21T22:07:23.466 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/cpufreq/p4-clockmod.ko 2024-01-21T22:07:23.466 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-01-21T22:07:23.466 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/cpufreq/powernow-k8.ko 2024-01-21T22:07:23.466 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/cpufreq/speedstep-lib.ko 2024-01-21T22:07:23.466 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/crypto 2024-01-21T22:07:23.466 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/crypto/ccp 2024-01-21T22:07:23.466 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-01-21T22:07:23.466 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/crypto/ccp/ccp.ko 2024-01-21T22:07:23.466 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/crypto/intel 2024-01-21T22:07:23.466 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/crypto/intel/qat 2024-01-21T22:07:23.466 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/crypto/intel/qat/qat_common 2024-01-21T22:07:23.466 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-01-21T22:07:23.466 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-01-21T22:07:23.467 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-01-21T22:07:23.467 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/crypto/padlock-aes.ko 2024-01-21T22:07:23.467 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/crypto/padlock-sha.ko 2024-01-21T22:07:23.467 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/crypto/virtio 2024-01-21T22:07:23.467 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-01-21T22:07:23.467 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/dca 2024-01-21T22:07:23.467 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/dca/dca.ko 2024-01-21T22:07:23.467 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/devfreq 2024-01-21T22:07:23.467 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-01-21T22:07:23.467 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/dma 2024-01-21T22:07:23.467 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/dma/dw 2024-01-21T22:07:23.467 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/dma/dw/dw_dmac.ko 2024-01-21T22:07:23.467 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/dma/ioat 2024-01-21T22:07:23.467 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/dma/ioat/ioatdma.ko 2024-01-21T22:07:23.467 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/edac 2024-01-21T22:07:23.468 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/edac/amd64_edac.ko 2024-01-21T22:07:23.468 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/edac/e752x_edac.ko 2024-01-21T22:07:23.468 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/edac/edac_mce_amd.ko 2024-01-21T22:07:23.468 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/edac/i3000_edac.ko 2024-01-21T22:07:23.468 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/edac/i3200_edac.ko 2024-01-21T22:07:23.468 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/edac/i5100_edac.ko 2024-01-21T22:07:23.468 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/edac/i5400_edac.ko 2024-01-21T22:07:23.468 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/edac/i7300_edac.ko 2024-01-21T22:07:23.468 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/edac/i7core_edac.ko 2024-01-21T22:07:23.468 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/edac/i82975x_edac.ko 2024-01-21T22:07:23.468 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/edac/ie31200_edac.ko 2024-01-21T22:07:23.468 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/edac/sb_edac.ko 2024-01-21T22:07:23.468 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/edac/x38_edac.ko 2024-01-21T22:07:23.468 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/firmware 2024-01-21T22:07:23.469 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/firmware/edd.ko 2024-01-21T22:07:23.469 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/firmware/iscsi_ibft.ko 2024-01-21T22:07:23.469 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/gpio 2024-01-21T22:07:23.469 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/gpio/gpio-ich.ko 2024-01-21T22:07:23.469 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/gpio/gpio-viperboard.ko 2024-01-21T22:07:23.469 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid 2024-01-21T22:07:23.469 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-appleir.ko 2024-01-21T22:07:23.469 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-aureal.ko 2024-01-21T22:07:23.469 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-axff.ko 2024-01-21T22:07:23.469 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-dr.ko 2024-01-21T22:07:23.469 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-elecom.ko 2024-01-21T22:07:23.469 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-elo.ko 2024-01-21T22:07:23.469 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-emsff.ko 2024-01-21T22:07:23.469 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-gaff.ko 2024-01-21T22:07:23.469 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-gt683r.ko 2024-01-21T22:07:23.469 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-gyration.ko 2024-01-21T22:07:23.470 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-holtek-kbd.ko 2024-01-21T22:07:23.470 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-holtek-mouse.ko 2024-01-21T22:07:23.470 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-holtekff.ko 2024-01-21T22:07:23.470 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-hyperv.ko 2024-01-21T22:07:23.470 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-icade.ko 2024-01-21T22:07:23.470 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-keytouch.ko 2024-01-21T22:07:23.470 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-kye.ko 2024-01-21T22:07:23.470 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-lcpower.ko 2024-01-21T22:07:23.470 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-led.ko 2024-01-21T22:07:23.470 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-lenovo.ko 2024-01-21T22:07:23.470 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-logitech-dj.ko 2024-01-21T22:07:23.470 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-01-21T22:07:23.470 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-multitouch.ko 2024-01-21T22:07:23.470 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-ortek.ko 2024-01-21T22:07:23.470 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-penmount.ko 2024-01-21T22:07:23.470 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-petalynx.ko 2024-01-21T22:07:23.471 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-picolcd.ko 2024-01-21T22:07:23.471 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-pl.ko 2024-01-21T22:07:23.471 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-plantronics.ko 2024-01-21T22:07:23.471 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-primax.ko 2024-01-21T22:07:23.471 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-rmi.ko 2024-01-21T22:07:23.471 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-roccat-arvo.ko 2024-01-21T22:07:23.471 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-roccat-common.ko 2024-01-21T22:07:23.471 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-roccat-isku.ko 2024-01-21T22:07:23.471 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-roccat-kone.ko 2024-01-21T22:07:23.471 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-01-21T22:07:23.471 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-roccat-konepure.ko 2024-01-21T22:07:23.471 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-01-21T22:07:23.471 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-roccat-lua.ko 2024-01-21T22:07:23.471 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-roccat-pyra.ko 2024-01-21T22:07:23.471 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-roccat-ryos.ko 2024-01-21T22:07:23.471 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-roccat-savu.ko 2024-01-21T22:07:23.471 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-roccat.ko 2024-01-21T22:07:23.472 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-saitek.ko 2024-01-21T22:07:23.472 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-samsung.ko 2024-01-21T22:07:23.472 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-sensor-hub.ko 2024-01-21T22:07:23.472 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-sjoy.ko 2024-01-21T22:07:23.472 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-sony.ko 2024-01-21T22:07:23.472 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-speedlink.ko 2024-01-21T22:07:23.472 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-steelseries.ko 2024-01-21T22:07:23.472 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-sunplus.ko 2024-01-21T22:07:23.472 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-thrustmaster.ko 2024-01-21T22:07:23.472 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-tivo.ko 2024-01-21T22:07:23.472 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-tmff.ko 2024-01-21T22:07:23.472 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-topseed.ko 2024-01-21T22:07:23.472 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-twinhan.ko 2024-01-21T22:07:23.472 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-uclogic.ko 2024-01-21T22:07:23.473 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-waltop.ko 2024-01-21T22:07:23.473 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-wiimote.ko 2024-01-21T22:07:23.473 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-xinmo.ko 2024-01-21T22:07:23.473 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-zpff.ko 2024-01-21T22:07:23.473 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/hid-zydacron.ko 2024-01-21T22:07:23.473 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/uhid.ko 2024-01-21T22:07:23.473 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hid/wacom.ko 2024-01-21T22:07:23.473 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hv 2024-01-21T22:07:23.473 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hv/hv_balloon.ko 2024-01-21T22:07:23.473 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hv/hv_utils.ko 2024-01-21T22:07:23.473 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hv/hv_vmbus.ko 2024-01-21T22:07:23.473 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon 2024-01-21T22:07:23.473 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/abituguru.ko 2024-01-21T22:07:23.473 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/abituguru3.ko 2024-01-21T22:07:23.473 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/acpi_power_meter.ko 2024-01-21T22:07:23.473 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/ad7414.ko 2024-01-21T22:07:23.474 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/ad7418.ko 2024-01-21T22:07:23.474 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/adc128d818.ko 2024-01-21T22:07:23.474 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/adm1025.ko 2024-01-21T22:07:23.474 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/adm1026.ko 2024-01-21T22:07:23.474 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/adm1029.ko 2024-01-21T22:07:23.474 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/adm1031.ko 2024-01-21T22:07:23.474 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/adm9240.ko 2024-01-21T22:07:23.474 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/ads7828.ko 2024-01-21T22:07:23.474 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/adt7410.ko 2024-01-21T22:07:23.474 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/adt7411.ko 2024-01-21T22:07:23.474 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/adt7462.ko 2024-01-21T22:07:23.474 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/adt7470.ko 2024-01-21T22:07:23.474 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/adt7475.ko 2024-01-21T22:07:23.474 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/adt7x10.ko 2024-01-21T22:07:23.474 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/amc6821.ko 2024-01-21T22:07:23.475 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/applesmc.ko 2024-01-21T22:07:23.475 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/asb100.ko 2024-01-21T22:07:23.475 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/asc7621.ko 2024-01-21T22:07:23.475 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/asus_atk0110.ko 2024-01-21T22:07:23.475 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/atxp1.ko 2024-01-21T22:07:23.475 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/coretemp.ko 2024-01-21T22:07:23.475 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-01-21T22:07:23.475 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/dme1737.ko 2024-01-21T22:07:23.475 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/ds1621.ko 2024-01-21T22:07:23.475 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/ds620.ko 2024-01-21T22:07:23.475 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/emc1403.ko 2024-01-21T22:07:23.475 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/emc6w201.ko 2024-01-21T22:07:23.475 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/f71805f.ko 2024-01-21T22:07:23.475 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/f71882fg.ko 2024-01-21T22:07:23.475 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/f75375s.ko 2024-01-21T22:07:23.476 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/fam15h_power.ko 2024-01-21T22:07:23.476 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/fschmd.ko 2024-01-21T22:07:23.476 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/g760a.ko 2024-01-21T22:07:23.476 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/g762.ko 2024-01-21T22:07:23.476 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/gl518sm.ko 2024-01-21T22:07:23.476 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/gl520sm.ko 2024-01-21T22:07:23.476 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/hwmon-vid.ko 2024-01-21T22:07:23.476 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/i5500_temp.ko 2024-01-21T22:07:23.476 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/i5k_amb.ko 2024-01-21T22:07:23.476 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/ibmaem.ko 2024-01-21T22:07:23.476 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/ibmpex.ko 2024-01-21T22:07:23.476 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/ina209.ko 2024-01-21T22:07:23.476 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/ina2xx.ko 2024-01-21T22:07:23.476 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/it87.ko 2024-01-21T22:07:23.477 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/k10temp.ko 2024-01-21T22:07:23.477 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/k8temp.ko 2024-01-21T22:07:23.477 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/lineage-pem.ko 2024-01-21T22:07:23.477 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/lm63.ko 2024-01-21T22:07:23.477 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/lm73.ko 2024-01-21T22:07:23.477 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/lm75.ko 2024-01-21T22:07:23.477 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/lm77.ko 2024-01-21T22:07:23.477 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/lm78.ko 2024-01-21T22:07:23.477 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/lm80.ko 2024-01-21T22:07:23.477 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/lm83.ko 2024-01-21T22:07:23.477 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/lm85.ko 2024-01-21T22:07:23.477 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/lm87.ko 2024-01-21T22:07:23.477 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/lm90.ko 2024-01-21T22:07:23.477 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/lm92.ko 2024-01-21T22:07:23.477 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/lm93.ko 2024-01-21T22:07:23.477 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/lm95234.ko 2024-01-21T22:07:23.478 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/lm95241.ko 2024-01-21T22:07:23.478 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/lm95245.ko 2024-01-21T22:07:23.478 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/ltc2945.ko 2024-01-21T22:07:23.478 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/ltc4151.ko 2024-01-21T22:07:23.478 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/ltc4215.ko 2024-01-21T22:07:23.478 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/ltc4222.ko 2024-01-21T22:07:23.478 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/ltc4245.ko 2024-01-21T22:07:23.478 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/ltc4260.ko 2024-01-21T22:07:23.478 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/ltc4261.ko 2024-01-21T22:07:23.478 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/max16065.ko 2024-01-21T22:07:23.478 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/max1619.ko 2024-01-21T22:07:23.478 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/max1668.ko 2024-01-21T22:07:23.478 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/max197.ko 2024-01-21T22:07:23.478 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/max6639.ko 2024-01-21T22:07:23.478 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/max6650.ko 2024-01-21T22:07:23.479 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/max6697.ko 2024-01-21T22:07:23.479 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/mcp3021.ko 2024-01-21T22:07:23.479 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/nct6683.ko 2024-01-21T22:07:23.479 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/nct6775-core.ko 2024-01-21T22:07:23.479 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/nct6775.ko 2024-01-21T22:07:23.479 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/nct7802.ko 2024-01-21T22:07:23.479 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/pc87360.ko 2024-01-21T22:07:23.479 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/pc87427.ko 2024-01-21T22:07:23.479 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/pcf8591.ko 2024-01-21T22:07:23.479 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/pmbus 2024-01-21T22:07:23.479 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-01-21T22:07:23.479 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-01-21T22:07:23.479 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-01-21T22:07:23.479 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/pmbus/max16064.ko 2024-01-21T22:07:23.479 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/pmbus/max34440.ko 2024-01-21T22:07:23.480 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/pmbus/max8688.ko 2024-01-21T22:07:23.480 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-01-21T22:07:23.480 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-01-21T22:07:23.480 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-01-21T22:07:23.480 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-01-21T22:07:23.480 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-01-21T22:07:23.480 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-01-21T22:07:23.480 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/powr1220.ko 2024-01-21T22:07:23.480 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/sch5627.ko 2024-01-21T22:07:23.480 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/sch5636.ko 2024-01-21T22:07:23.480 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/sch56xx-common.ko 2024-01-21T22:07:23.480 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/sht15.ko 2024-01-21T22:07:23.480 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/sht21.ko 2024-01-21T22:07:23.480 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/shtc1.ko 2024-01-21T22:07:23.481 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/sis5595.ko 2024-01-21T22:07:23.481 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/smsc47b397.ko 2024-01-21T22:07:23.481 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/smsc47m1.ko 2024-01-21T22:07:23.481 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/smsc47m192.ko 2024-01-21T22:07:23.481 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/thmc50.ko 2024-01-21T22:07:23.481 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/tmp102.ko 2024-01-21T22:07:23.481 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/tmp103.ko 2024-01-21T22:07:23.481 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/tmp401.ko 2024-01-21T22:07:23.481 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/tmp421.ko 2024-01-21T22:07:23.481 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/via-cputemp.ko 2024-01-21T22:07:23.481 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/via686a.ko 2024-01-21T22:07:23.481 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/vt1211.ko 2024-01-21T22:07:23.481 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/vt8231.ko 2024-01-21T22:07:23.483 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/w83627ehf.ko 2024-01-21T22:07:23.483 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/w83627hf.ko 2024-01-21T22:07:23.483 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/w83781d.ko 2024-01-21T22:07:23.483 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/w83791d.ko 2024-01-21T22:07:23.483 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/w83792d.ko 2024-01-21T22:07:23.484 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/w83793.ko 2024-01-21T22:07:23.484 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/w83795.ko 2024-01-21T22:07:23.484 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/w83l785ts.ko 2024-01-21T22:07:23.484 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/hwmon/w83l786ng.ko 2024-01-21T22:07:23.484 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c 2024-01-21T22:07:23.484 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/algos 2024-01-21T22:07:23.484 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-01-21T22:07:23.484 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-01-21T22:07:23.484 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses 2024-01-21T22:07:23.484 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-01-21T22:07:23.484 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-01-21T22:07:23.484 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-01-21T22:07:23.484 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-01-21T22:07:23.484 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-01-21T22:07:23.484 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-01-21T22:07:23.485 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-01-21T22:07:23.485 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-01-21T22:07:23.485 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses/i2c-i801.ko 2024-01-21T22:07:23.485 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses/i2c-isch.ko 2024-01-21T22:07:23.485 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-01-21T22:07:23.485 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-01-21T22:07:23.485 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-01-21T22:07:23.485 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses/i2c-parport.ko 2024-01-21T22:07:23.485 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-01-21T22:07:23.485 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-01-21T22:07:23.485 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-01-21T22:07:23.485 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-01-21T22:07:23.485 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-01-21T22:07:23.485 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-01-21T22:07:23.485 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses/i2c-via.ko 2024-01-21T22:07:23.486 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-01-21T22:07:23.486 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-01-21T22:07:23.486 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/i2c-dev.ko 2024-01-21T22:07:23.486 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/i2c-mux.ko 2024-01-21T22:07:23.486 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-01-21T22:07:23.486 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/i2c-smbus.ko 2024-01-21T22:07:23.486 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/i2c/i2c-stub.ko 2024-01-21T22:07:23.486 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband 2024-01-21T22:07:23.486 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/core 2024-01-21T22:07:23.486 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/core/ib_cm.ko 2024-01-21T22:07:23.486 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/core/ib_core.ko 2024-01-21T22:07:23.486 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/core/ib_umad.ko 2024-01-21T22:07:23.486 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-01-21T22:07:23.486 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/core/iw_cm.ko 2024-01-21T22:07:23.486 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/core/rdma_cm.ko 2024-01-21T22:07:23.487 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-01-21T22:07:23.487 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/hw 2024-01-21T22:07:23.487 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/hw/cxgb4 2024-01-21T22:07:23.487 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-01-21T22:07:23.487 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/hw/mlx4 2024-01-21T22:07:23.487 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-01-21T22:07:23.487 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/hw/mlx5 2024-01-21T22:07:23.487 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-01-21T22:07:23.487 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/hw/mthca 2024-01-21T22:07:23.487 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-01-21T22:07:23.487 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/ulp 2024-01-21T22:07:23.487 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/ulp/ipoib 2024-01-21T22:07:23.487 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-01-21T22:07:23.487 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/ulp/iser 2024-01-21T22:07:23.487 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-01-21T22:07:23.488 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/ulp/isert 2024-01-21T22:07:23.488 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-01-21T22:07:23.488 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/ulp/srp 2024-01-21T22:07:23.488 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-01-21T22:07:23.488 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/ulp/srpt 2024-01-21T22:07:23.488 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-01-21T22:07:23.488 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input 2024-01-21T22:07:23.488 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/gameport 2024-01-21T22:07:23.488 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-01-21T22:07:23.488 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/gameport/fm801-gp.ko 2024-01-21T22:07:23.488 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/gameport/gameport.ko 2024-01-21T22:07:23.488 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/joydev.ko 2024-01-21T22:07:23.488 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/keyboard 2024-01-21T22:07:23.488 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/keyboard/gpio_keys.ko 2024-01-21T22:07:23.489 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/mouse 2024-01-21T22:07:23.489 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/mouse/appletouch.ko 2024-01-21T22:07:23.489 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/mouse/bcm5974.ko 2024-01-21T22:07:23.489 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/mouse/cyapatp.ko 2024-01-21T22:07:23.489 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/mouse/elan_i2c.ko 2024-01-21T22:07:23.489 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/mouse/sermouse.ko 2024-01-21T22:07:23.489 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-01-21T22:07:23.489 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/mouse/synaptics_usb.ko 2024-01-21T22:07:23.489 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/mouse/vsxxxaa.ko 2024-01-21T22:07:23.489 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/rmi4 2024-01-21T22:07:23.489 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/rmi4/rmi_core.ko 2024-01-21T22:07:23.489 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/serio 2024-01-21T22:07:23.489 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/serio/altera_ps2.ko 2024-01-21T22:07:23.489 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/serio/arc_ps2.ko 2024-01-21T22:07:23.489 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-01-21T22:07:23.490 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/serio/serio_raw.ko 2024-01-21T22:07:23.490 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/input/sparse-keymap.ko 2024-01-21T22:07:23.490 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/leds 2024-01-21T22:07:23.490 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/leds/leds-blinkm.ko 2024-01-21T22:07:23.490 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/leds/leds-lm3530.ko 2024-01-21T22:07:23.490 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/leds/leds-lp3944.ko 2024-01-21T22:07:23.490 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/leds/leds-ss4200.ko 2024-01-21T22:07:23.490 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/leds/trigger 2024-01-21T22:07:23.490 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-01-21T22:07:23.490 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-01-21T22:07:23.490 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-01-21T22:07:23.490 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-01-21T22:07:23.490 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-01-21T22:07:23.490 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-01-21T22:07:23.490 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-01-21T22:07:23.491 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-01-21T22:07:23.491 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-01-21T22:07:23.491 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md 2024-01-21T22:07:23.491 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/bcache 2024-01-21T22:07:23.491 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/bcache/bcache.ko 2024-01-21T22:07:23.491 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/dm-bio-prison.ko 2024-01-21T22:07:23.491 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/dm-cache-smq.ko 2024-01-21T22:07:23.491 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/dm-cache.ko 2024-01-21T22:07:23.491 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/dm-crypt.ko 2024-01-21T22:07:23.491 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/dm-delay.ko 2024-01-21T22:07:23.491 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/dm-flakey.ko 2024-01-21T22:07:23.491 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/dm-log-userspace.ko 2024-01-21T22:07:23.491 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/dm-multipath.ko 2024-01-21T22:07:23.491 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/dm-queue-length.ko 2024-01-21T22:07:23.491 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/dm-raid.ko 2024-01-21T22:07:23.492 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/dm-round-robin.ko 2024-01-21T22:07:23.492 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/dm-service-time.ko 2024-01-21T22:07:23.492 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/dm-switch.ko 2024-01-21T22:07:23.492 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/dm-thin-pool.ko 2024-01-21T22:07:23.492 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/dm-verity.ko 2024-01-21T22:07:23.492 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/faulty.ko 2024-01-21T22:07:23.492 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/linear.ko 2024-01-21T22:07:23.492 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/multipath.ko 2024-01-21T22:07:23.492 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/persistent-data 2024-01-21T22:07:23.492 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-01-21T22:07:23.492 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/raid0.ko 2024-01-21T22:07:23.492 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/raid1.ko 2024-01-21T22:07:23.492 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/raid10.ko 2024-01-21T22:07:23.492 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/md/raid456.ko 2024-01-21T22:07:23.493 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/message 2024-01-21T22:07:23.493 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/message/fusion 2024-01-21T22:07:23.493 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/message/fusion/mptbase.ko 2024-01-21T22:07:23.493 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/message/fusion/mptctl.ko 2024-01-21T22:07:23.493 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/message/fusion/mptfc.ko 2024-01-21T22:07:23.493 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/message/fusion/mptlan.ko 2024-01-21T22:07:23.493 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/message/fusion/mptsas.ko 2024-01-21T22:07:23.493 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/message/fusion/mptscsih.ko 2024-01-21T22:07:23.493 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/message/fusion/mptspi.ko 2024-01-21T22:07:23.493 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/mfd 2024-01-21T22:07:23.493 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/mfd/lpc_ich.ko 2024-01-21T22:07:23.493 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/mfd/lpc_sch.ko 2024-01-21T22:07:23.493 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/mfd/mfd-core.ko 2024-01-21T22:07:23.493 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/mfd/sm501.ko 2024-01-21T22:07:23.493 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/mfd/viperboard.ko 2024-01-21T22:07:23.494 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/mfd/vx855.ko 2024-01-21T22:07:23.494 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/mfd/wl1273-core.ko 2024-01-21T22:07:23.494 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc 2024-01-21T22:07:23.494 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/altera-stapl 2024-01-21T22:07:23.494 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-01-21T22:07:23.494 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/apds9802als.ko 2024-01-21T22:07:23.494 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/apds990x.ko 2024-01-21T22:07:23.494 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/bh1770glc.ko 2024-01-21T22:07:23.494 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/cb710 2024-01-21T22:07:23.494 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/cb710/cb710.ko 2024-01-21T22:07:23.494 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/eeprom 2024-01-21T22:07:23.494 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/eeprom/at24.ko 2024-01-21T22:07:23.494 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-01-21T22:07:23.494 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/eeprom/max6875.ko 2024-01-21T22:07:23.494 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/enclosure.ko 2024-01-21T22:07:23.495 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/hpilo.ko 2024-01-21T22:07:23.495 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/isl29003.ko 2024-01-21T22:07:23.495 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/isl29020.ko 2024-01-21T22:07:23.495 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/lis3lv02d 2024-01-21T22:07:23.495 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-01-21T22:07:23.495 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-01-21T22:07:23.495 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/mei 2024-01-21T22:07:23.495 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/mei/mei-me.ko 2024-01-21T22:07:23.495 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/mei/mei-txe.ko 2024-01-21T22:07:23.495 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/mei/mei.ko 2024-01-21T22:07:23.495 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/sgi-gru 2024-01-21T22:07:23.495 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/sgi-gru/gru.ko 2024-01-21T22:07:23.495 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/sgi-xp 2024-01-21T22:07:23.495 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/sgi-xp/xp.ko 2024-01-21T22:07:23.495 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/sgi-xp/xpc.ko 2024-01-21T22:07:23.496 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-01-21T22:07:23.496 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/tifm_7xx1.ko 2024-01-21T22:07:23.496 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/tifm_core.ko 2024-01-21T22:07:23.496 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/tsl2550.ko 2024-01-21T22:07:23.496 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/vmw_balloon.ko 2024-01-21T22:07:23.496 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/vmw_vmci 2024-01-21T22:07:23.496 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-01-21T22:07:23.496 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net 2024-01-21T22:07:23.496 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/bonding 2024-01-21T22:07:23.496 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/bonding/bonding.ko 2024-01-21T22:07:23.496 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/dsa 2024-01-21T22:07:23.496 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/dsa/b53 2024-01-21T22:07:23.496 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/dsa/b53/b53_common.ko 2024-01-21T22:07:23.496 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/dsa/bcm-sf2.ko 2024-01-21T22:07:23.496 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/dsa/mv88e6060.ko 2024-01-21T22:07:23.497 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/dsa/mv88e6xxx 2024-01-21T22:07:23.497 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-01-21T22:07:23.497 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/dummy.ko 2024-01-21T22:07:23.497 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/eql.ko 2024-01-21T22:07:23.497 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet 2024-01-21T22:07:23.497 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/3com 2024-01-21T22:07:23.497 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-01-21T22:07:23.497 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-01-21T22:07:23.497 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-01-21T22:07:23.497 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-01-21T22:07:23.497 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/8390 2024-01-21T22:07:23.497 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/8390/8390.ko 2024-01-21T22:07:23.497 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-01-21T22:07:23.497 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-01-21T22:07:23.497 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-01-21T22:07:23.498 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/adaptec 2024-01-21T22:07:23.498 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-01-21T22:07:23.498 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/agere 2024-01-21T22:07:23.498 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/agere/et131x.ko 2024-01-21T22:07:23.498 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/alteon 2024-01-21T22:07:23.498 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-01-21T22:07:23.498 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/altera 2024-01-21T22:07:23.498 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-01-21T22:07:23.498 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/amd 2024-01-21T22:07:23.498 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-01-21T22:07:23.498 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-01-21T22:07:23.498 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-01-21T22:07:23.498 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/atheros 2024-01-21T22:07:23.498 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/atheros/alx 2024-01-21T22:07:23.498 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-01-21T22:07:23.498 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/atheros/atl1c 2024-01-21T22:07:23.499 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-01-21T22:07:23.499 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/atheros/atl1e 2024-01-21T22:07:23.499 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-01-21T22:07:23.499 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/atheros/atlx 2024-01-21T22:07:23.499 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-01-21T22:07:23.499 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-01-21T22:07:23.499 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/broadcom 2024-01-21T22:07:23.499 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-01-21T22:07:23.499 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-01-21T22:07:23.499 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-01-21T22:07:23.499 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-01-21T22:07:23.499 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-01-21T22:07:23.499 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/broadcom/genet 2024-01-21T22:07:23.499 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-01-21T22:07:23.499 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-01-21T22:07:23.500 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/brocade 2024-01-21T22:07:23.500 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/brocade/bna 2024-01-21T22:07:23.500 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-01-21T22:07:23.500 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/chelsio 2024-01-21T22:07:23.500 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/chelsio/cxgb 2024-01-21T22:07:23.500 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-01-21T22:07:23.500 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-01-21T22:07:23.500 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-01-21T22:07:23.500 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-01-21T22:07:23.500 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-01-21T22:07:23.500 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-01-21T22:07:23.500 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-01-21T22:07:23.500 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-01-21T22:07:23.500 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-01-21T22:07:23.500 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/cisco 2024-01-21T22:07:23.501 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/cisco/enic 2024-01-21T22:07:23.501 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-01-21T22:07:23.501 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/dec 2024-01-21T22:07:23.501 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/dec/tulip 2024-01-21T22:07:23.501 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-01-21T22:07:23.501 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-01-21T22:07:23.501 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-01-21T22:07:23.501 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-01-21T22:07:23.501 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-01-21T22:07:23.501 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-01-21T22:07:23.501 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/dlink 2024-01-21T22:07:23.501 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-01-21T22:07:23.501 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-01-21T22:07:23.501 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/dnet.ko 2024-01-21T22:07:23.501 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/emulex 2024-01-21T22:07:23.502 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/emulex/benet 2024-01-21T22:07:23.502 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-01-21T22:07:23.502 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/ethoc.ko 2024-01-21T22:07:23.502 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/fealnx.ko 2024-01-21T22:07:23.502 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/intel 2024-01-21T22:07:23.502 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/intel/e100.ko 2024-01-21T22:07:23.502 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/intel/e1000 2024-01-21T22:07:23.502 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-01-21T22:07:23.502 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/intel/e1000e 2024-01-21T22:07:23.502 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-01-21T22:07:23.502 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/intel/fm10k 2024-01-21T22:07:23.502 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-01-21T22:07:23.502 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/intel/i40e 2024-01-21T22:07:23.502 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-01-21T22:07:23.502 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/intel/iavf 2024-01-21T22:07:23.503 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-01-21T22:07:23.503 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/intel/igb 2024-01-21T22:07:23.503 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-01-21T22:07:23.503 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/intel/igbvf 2024-01-21T22:07:23.503 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-01-21T22:07:23.503 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/intel/ixgbe 2024-01-21T22:07:23.503 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-01-21T22:07:23.503 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/intel/ixgbevf 2024-01-21T22:07:23.503 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-01-21T22:07:23.503 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/jme.ko 2024-01-21T22:07:23.503 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/marvell 2024-01-21T22:07:23.503 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-01-21T22:07:23.503 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/marvell/skge.ko 2024-01-21T22:07:23.503 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-01-21T22:07:23.503 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/mellanox 2024-01-21T22:07:23.504 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/mellanox/mlx4 2024-01-21T22:07:23.504 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-01-21T22:07:23.504 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-01-21T22:07:23.504 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/mellanox/mlx5 2024-01-21T22:07:23.504 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-01-21T22:07:23.504 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-01-21T22:07:23.504 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/micrel 2024-01-21T22:07:23.504 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-01-21T22:07:23.504 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/myricom 2024-01-21T22:07:23.504 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/myricom/myri10ge 2024-01-21T22:07:23.504 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-01-21T22:07:23.504 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/natsemi 2024-01-21T22:07:23.504 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-01-21T22:07:23.504 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-01-21T22:07:23.505 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/neterion 2024-01-21T22:07:23.505 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-01-21T22:07:23.505 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/nvidia 2024-01-21T22:07:23.505 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-01-21T22:07:23.505 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/packetengines 2024-01-21T22:07:23.505 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-01-21T22:07:23.505 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-01-21T22:07:23.505 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/qlogic 2024-01-21T22:07:23.505 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/qlogic/netxen 2024-01-21T22:07:23.505 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-01-21T22:07:23.505 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-01-21T22:07:23.505 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-01-21T22:07:23.505 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-01-21T22:07:23.505 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/rdc 2024-01-21T22:07:23.505 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-01-21T22:07:23.506 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/realtek 2024-01-21T22:07:23.506 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-01-21T22:07:23.506 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-01-21T22:07:23.506 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/realtek/atp.ko 2024-01-21T22:07:23.506 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-01-21T22:07:23.506 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/rocker 2024-01-21T22:07:23.506 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-01-21T22:07:23.506 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/sfc 2024-01-21T22:07:23.506 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-01-21T22:07:23.506 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/silan 2024-01-21T22:07:23.506 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-01-21T22:07:23.506 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/sis 2024-01-21T22:07:23.506 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/sis/sis190.ko 2024-01-21T22:07:23.506 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/sis/sis900.ko 2024-01-21T22:07:23.506 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/smsc 2024-01-21T22:07:23.506 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-01-21T22:07:23.507 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-01-21T22:07:23.507 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-01-21T22:07:23.507 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-01-21T22:07:23.507 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/stmicro 2024-01-21T22:07:23.507 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/stmicro/stmmac 2024-01-21T22:07:23.507 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-01-21T22:07:23.507 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-01-21T22:07:23.507 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/sun 2024-01-21T22:07:23.507 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/sun/cassini.ko 2024-01-21T22:07:23.507 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/sun/niu.ko 2024-01-21T22:07:23.507 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/sun/sungem.ko 2024-01-21T22:07:23.507 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-01-21T22:07:23.507 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/tehuti 2024-01-21T22:07:23.507 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-01-21T22:07:23.507 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/ti 2024-01-21T22:07:23.508 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/ti/tlan.ko 2024-01-21T22:07:23.508 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/via 2024-01-21T22:07:23.508 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-01-21T22:07:23.508 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-01-21T22:07:23.508 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/wiznet 2024-01-21T22:07:23.508 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-01-21T22:07:23.508 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-01-21T22:07:23.508 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/xircom 2024-01-21T22:07:23.508 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-01-21T22:07:23.508 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/geneve.ko 2024-01-21T22:07:23.508 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/hyperv 2024-01-21T22:07:23.508 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-01-21T22:07:23.508 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ieee802154 2024-01-21T22:07:23.508 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ieee802154/fakelb.ko 2024-01-21T22:07:23.508 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ifb.ko 2024-01-21T22:07:23.509 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ipvlan 2024-01-21T22:07:23.509 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ipvlan/ipvlan.ko 2024-01-21T22:07:23.509 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/macvlan.ko 2024-01-21T22:07:23.509 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/macvtap.ko 2024-01-21T22:07:23.509 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/mdio 2024-01-21T22:07:23.509 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/mdio.ko 2024-01-21T22:07:23.509 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-01-21T22:07:23.509 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-01-21T22:07:23.509 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/mdio/mdio-regmap.ko 2024-01-21T22:07:23.509 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/mii.ko 2024-01-21T22:07:23.509 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/net_failover.ko 2024-01-21T22:07:23.509 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/netconsole.ko 2024-01-21T22:07:23.509 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/nlmon.ko 2024-01-21T22:07:23.509 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/pcs 2024-01-21T22:07:23.509 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/pcs/pcs-lynx.ko 2024-01-21T22:07:23.510 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-01-21T22:07:23.510 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/phy 2024-01-21T22:07:23.510 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/phy/amd.ko 2024-01-21T22:07:23.510 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-01-21T22:07:23.510 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-01-21T22:07:23.510 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/phy/bcm7xxx.ko 2024-01-21T22:07:23.510 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/phy/bcm87xx.ko 2024-01-21T22:07:23.510 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/phy/broadcom.ko 2024-01-21T22:07:23.510 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/phy/cicada.ko 2024-01-21T22:07:23.510 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/phy/davicom.ko 2024-01-21T22:07:23.510 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/phy/dp83640.ko 2024-01-21T22:07:23.510 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/phy/et1011c.ko 2024-01-21T22:07:23.510 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/phy/icplus.ko 2024-01-21T22:07:23.510 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/phy/lxt.ko 2024-01-21T22:07:23.510 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/phy/marvell.ko 2024-01-21T22:07:23.511 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/phy/micrel.ko 2024-01-21T22:07:23.511 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/phy/national.ko 2024-01-21T22:07:23.511 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/phy/phylink.ko 2024-01-21T22:07:23.511 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/phy/qsemi.ko 2024-01-21T22:07:23.511 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/phy/realtek.ko 2024-01-21T22:07:23.511 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/phy/smsc.ko 2024-01-21T22:07:23.511 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/phy/ste10Xp.ko 2024-01-21T22:07:23.511 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/phy/vitesse.ko 2024-01-21T22:07:23.511 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ppp 2024-01-21T22:07:23.511 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ppp/bsd_comp.ko 2024-01-21T22:07:23.511 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ppp/ppp_async.ko 2024-01-21T22:07:23.511 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ppp/ppp_deflate.ko 2024-01-21T22:07:23.511 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ppp/ppp_generic.ko 2024-01-21T22:07:23.511 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ppp/ppp_mppe.ko 2024-01-21T22:07:23.512 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ppp/ppp_synctty.ko 2024-01-21T22:07:23.512 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ppp/pppoe.ko 2024-01-21T22:07:23.512 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ppp/pppox.ko 2024-01-21T22:07:23.512 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/ppp/pptp.ko 2024-01-21T22:07:23.512 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/slip 2024-01-21T22:07:23.512 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/slip/slhc.ko 2024-01-21T22:07:23.512 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/slip/slip.ko 2024-01-21T22:07:23.512 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/sungem_phy.ko 2024-01-21T22:07:23.512 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/tap.ko 2024-01-21T22:07:23.512 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/team 2024-01-21T22:07:23.512 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/team/team.ko 2024-01-21T22:07:23.512 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/team/team_mode_activebackup.ko 2024-01-21T22:07:23.512 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/team/team_mode_broadcast.ko 2024-01-21T22:07:23.512 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-01-21T22:07:23.512 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/team/team_mode_random.ko 2024-01-21T22:07:23.513 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-01-21T22:07:23.513 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/tun.ko 2024-01-21T22:07:23.513 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/veth.ko 2024-01-21T22:07:23.513 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/virtio_net.ko 2024-01-21T22:07:23.513 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/vmxnet3 2024-01-21T22:07:23.513 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-01-21T22:07:23.513 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/vxlan 2024-01-21T22:07:23.513 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/vxlan/vxlan.ko 2024-01-21T22:07:23.513 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/xen-netback 2024-01-21T22:07:23.513 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/xen-netback/xen-netback.ko 2024-01-21T22:07:23.513 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/net/xen-netfront.ko 2024-01-21T22:07:23.513 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ntb 2024-01-21T22:07:23.513 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ntb/ntb.ko 2024-01-21T22:07:23.513 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/nvme 2024-01-21T22:07:23.513 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/nvme/host 2024-01-21T22:07:23.514 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/nvme/host/nvme-core.ko 2024-01-21T22:07:23.514 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/nvme/host/nvme.ko 2024-01-21T22:07:23.514 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/parport 2024-01-21T22:07:23.514 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/parport/parport.ko 2024-01-21T22:07:23.514 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/parport/parport_cs.ko 2024-01-21T22:07:23.514 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/parport/parport_pc.ko 2024-01-21T22:07:23.514 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/parport/parport_serial.ko 2024-01-21T22:07:23.514 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/pci 2024-01-21T22:07:23.514 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/pci/hotplug 2024-01-21T22:07:23.514 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-01-21T22:07:23.514 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/pci/pcie 2024-01-21T22:07:23.514 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/pci/pcie/aer_inject.ko 2024-01-21T22:07:23.514 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/pci/xen-pcifront.ko 2024-01-21T22:07:23.514 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/pcmcia 2024-01-21T22:07:23.514 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/pcmcia/i82092.ko 2024-01-21T22:07:23.514 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/pcmcia/pd6729.ko 2024-01-21T22:07:23.515 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/pcmcia/yenta_socket.ko 2024-01-21T22:07:23.515 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/pinctrl 2024-01-21T22:07:23.515 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/pinctrl/intel 2024-01-21T22:07:23.515 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-01-21T22:07:23.515 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform 2024-01-21T22:07:23.515 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86 2024-01-21T22:07:23.515 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/acer-wmi.ko 2024-01-21T22:07:23.515 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/acerhdf.ko 2024-01-21T22:07:23.515 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/apple-gmux.ko 2024-01-21T22:07:23.515 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/asus-laptop.ko 2024-01-21T22:07:23.515 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-01-21T22:07:23.515 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/asus-wmi.ko 2024-01-21T22:07:23.515 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/classmate-laptop.ko 2024-01-21T22:07:23.515 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-01-21T22:07:23.515 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-01-21T22:07:23.516 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-01-21T22:07:23.516 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-01-21T22:07:23.516 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/hdaps.ko 2024-01-21T22:07:23.516 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/intel 2024-01-21T22:07:23.516 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-01-21T22:07:23.516 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/intel_ips.ko 2024-01-21T22:07:23.516 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/msi-wmi.ko 2024-01-21T22:07:23.516 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/mxm-wmi.ko 2024-01-21T22:07:23.516 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-01-21T22:07:23.516 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/samsung-laptop.ko 2024-01-21T22:07:23.516 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/samsung-q10.ko 2024-01-21T22:07:23.516 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/topstar-laptop.ko 2024-01-21T22:07:23.516 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-01-21T22:07:23.516 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/toshiba_haps.ko 2024-01-21T22:07:23.516 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/wmi-bmof.ko 2024-01-21T22:07:23.517 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/platform/x86/wmi.ko 2024-01-21T22:07:23.517 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/powercap 2024-01-21T22:07:23.517 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/powercap/intel_rapl_common.ko 2024-01-21T22:07:23.517 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/powercap/intel_rapl_msr.ko 2024-01-21T22:07:23.517 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/pps 2024-01-21T22:07:23.517 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/pps/clients 2024-01-21T22:07:23.517 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/pps/clients/pps-gpio.ko 2024-01-21T22:07:23.517 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/pps/clients/pps-ldisc.ko 2024-01-21T22:07:23.517 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/pps/clients/pps_parport.ko 2024-01-21T22:07:23.517 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/pps/pps_core.ko 2024-01-21T22:07:23.517 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ptp 2024-01-21T22:07:23.517 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ptp/ptp.ko 2024-01-21T22:07:23.517 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ptp/ptp_kvm.ko 2024-01-21T22:07:23.517 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc 2024-01-21T22:07:23.517 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-bq32k.ko 2024-01-21T22:07:23.518 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-ds1286.ko 2024-01-21T22:07:23.518 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-ds1307.ko 2024-01-21T22:07:23.518 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-ds1374.ko 2024-01-21T22:07:23.518 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-ds1511.ko 2024-01-21T22:07:23.518 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-ds1553.ko 2024-01-21T22:07:23.518 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-ds1672.ko 2024-01-21T22:07:23.518 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-ds1742.ko 2024-01-21T22:07:23.518 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-ds2404.ko 2024-01-21T22:07:23.518 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-ds3232.ko 2024-01-21T22:07:23.518 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-em3027.ko 2024-01-21T22:07:23.518 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-fm3130.ko 2024-01-21T22:07:23.518 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-isl12022.ko 2024-01-21T22:07:23.518 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-isl1208.ko 2024-01-21T22:07:23.518 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-m41t80.ko 2024-01-21T22:07:23.518 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-m48t35.ko 2024-01-21T22:07:23.519 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-m48t59.ko 2024-01-21T22:07:23.519 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-max6900.ko 2024-01-21T22:07:23.519 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-msm6242.ko 2024-01-21T22:07:23.519 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-pcf2127.ko 2024-01-21T22:07:23.519 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-pcf85063.ko 2024-01-21T22:07:23.519 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-pcf8523.ko 2024-01-21T22:07:23.519 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-pcf8563.ko 2024-01-21T22:07:23.519 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-pcf8583.ko 2024-01-21T22:07:23.519 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-rp5c01.ko 2024-01-21T22:07:23.519 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-rs5c372.ko 2024-01-21T22:07:23.519 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-01-21T22:07:23.519 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-rx8025.ko 2024-01-21T22:07:23.519 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-rx8581.ko 2024-01-21T22:07:23.519 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-01-21T22:07:23.519 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/rtc/rtc-x1205.ko 2024-01-21T22:07:23.519 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi 2024-01-21T22:07:23.520 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/3w-9xxx.ko 2024-01-21T22:07:23.520 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/3w-sas.ko 2024-01-21T22:07:23.520 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/3w-xxxx.ko 2024-01-21T22:07:23.520 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/BusLogic.ko 2024-01-21T22:07:23.520 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/a100u2w.ko 2024-01-21T22:07:23.520 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/aacraid 2024-01-21T22:07:23.520 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/aacraid/aacraid.ko 2024-01-21T22:07:23.520 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/advansys.ko 2024-01-21T22:07:23.520 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/aic7xxx 2024-01-21T22:07:23.520 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-01-21T22:07:23.520 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-01-21T22:07:23.520 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/am53c974.ko 2024-01-21T22:07:23.520 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/arcmsr 2024-01-21T22:07:23.520 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-01-21T22:07:23.521 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/atp870u.ko 2024-01-21T22:07:23.521 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/be2iscsi 2024-01-21T22:07:23.521 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-01-21T22:07:23.521 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/bfa 2024-01-21T22:07:23.521 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/bfa/bfa.ko 2024-01-21T22:07:23.521 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/bnx2fc 2024-01-21T22:07:23.521 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-01-21T22:07:23.521 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/bnx2i 2024-01-21T22:07:23.521 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-01-21T22:07:23.521 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/ch.ko 2024-01-21T22:07:23.521 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/csiostor 2024-01-21T22:07:23.521 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/csiostor/csiostor.ko 2024-01-21T22:07:23.521 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/cxgbi 2024-01-21T22:07:23.521 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/cxgbi/cxgb3i 2024-01-21T22:07:23.521 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-01-21T22:07:23.522 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/cxgbi/cxgb4i 2024-01-21T22:07:23.522 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-01-21T22:07:23.522 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-01-21T22:07:23.522 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/dc395x.ko 2024-01-21T22:07:23.522 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/device_handler 2024-01-21T22:07:23.522 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-01-21T22:07:23.522 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-01-21T22:07:23.522 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-01-21T22:07:23.522 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-01-21T22:07:23.522 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/dmx3191d.ko 2024-01-21T22:07:23.522 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/esas2r 2024-01-21T22:07:23.522 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/esas2r/esas2r.ko 2024-01-21T22:07:23.522 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/esp_scsi.ko 2024-01-21T22:07:23.522 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/fcoe 2024-01-21T22:07:23.522 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/fcoe/fcoe.ko 2024-01-21T22:07:23.523 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-01-21T22:07:23.523 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/fnic 2024-01-21T22:07:23.523 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/fnic/fnic.ko 2024-01-21T22:07:23.523 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/hpsa.ko 2024-01-21T22:07:23.523 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/hptiop.ko 2024-01-21T22:07:23.523 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/hv_storvsc.ko 2024-01-21T22:07:23.523 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/initio.ko 2024-01-21T22:07:23.523 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/ipr.ko 2024-01-21T22:07:23.523 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/ips.ko 2024-01-21T22:07:23.523 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/isci 2024-01-21T22:07:23.523 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/isci/isci.ko 2024-01-21T22:07:23.523 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-01-21T22:07:23.523 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/iscsi_tcp.ko 2024-01-21T22:07:23.523 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/libfc 2024-01-21T22:07:23.523 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/libfc/libfc.ko 2024-01-21T22:07:23.524 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/libiscsi.ko 2024-01-21T22:07:23.524 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/libiscsi_tcp.ko 2024-01-21T22:07:23.524 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/libsas 2024-01-21T22:07:23.524 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/libsas/libsas.ko 2024-01-21T22:07:23.524 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/lpfc 2024-01-21T22:07:23.524 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/lpfc/lpfc.ko 2024-01-21T22:07:23.524 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/megaraid 2024-01-21T22:07:23.524 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/megaraid.ko 2024-01-21T22:07:23.524 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-01-21T22:07:23.524 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-01-21T22:07:23.524 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-01-21T22:07:23.524 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/mpt3sas 2024-01-21T22:07:23.524 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-01-21T22:07:23.524 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/mvsas 2024-01-21T22:07:23.525 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/mvsas/mvsas.ko 2024-01-21T22:07:23.525 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/mvumi.ko 2024-01-21T22:07:23.525 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/pm8001 2024-01-21T22:07:23.525 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-01-21T22:07:23.525 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/pmcraid.ko 2024-01-21T22:07:23.525 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/qla1280.ko 2024-01-21T22:07:23.525 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/qla2xxx 2024-01-21T22:07:23.525 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-01-21T22:07:23.525 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-01-21T22:07:23.525 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/qla4xxx 2024-01-21T22:07:23.525 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-01-21T22:07:23.525 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/raid_class.ko 2024-01-21T22:07:23.525 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/scsi_debug.ko 2024-01-21T22:07:23.525 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/scsi_transport_fc.ko 2024-01-21T22:07:23.525 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-01-21T22:07:23.525 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/scsi_transport_sas.ko 2024-01-21T22:07:23.526 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/scsi_transport_spi.ko 2024-01-21T22:07:23.526 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/scsi_transport_srp.ko 2024-01-21T22:07:23.526 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/ses.ko 2024-01-21T22:07:23.526 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/st.ko 2024-01-21T22:07:23.526 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/stex.ko 2024-01-21T22:07:23.526 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/sym53c8xx_2 2024-01-21T22:07:23.526 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-01-21T22:07:23.526 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/virtio_scsi.ko 2024-01-21T22:07:23.526 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/vmw_pvscsi.ko 2024-01-21T22:07:23.526 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/scsi/wd719x.ko 2024-01-21T22:07:23.526 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ssb 2024-01-21T22:07:23.526 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ssb/ssb.ko 2024-01-21T22:07:23.526 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/target 2024-01-21T22:07:23.526 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/target/iscsi 2024-01-21T22:07:23.527 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-01-21T22:07:23.527 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/target/loopback 2024-01-21T22:07:23.527 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/target/loopback/tcm_loop.ko 2024-01-21T22:07:23.527 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/target/target_core_file.ko 2024-01-21T22:07:23.527 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/target/target_core_iblock.ko 2024-01-21T22:07:23.527 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/target/target_core_mod.ko 2024-01-21T22:07:23.527 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/target/target_core_pscsi.ko 2024-01-21T22:07:23.527 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/target/target_core_user.ko 2024-01-21T22:07:23.527 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/target/tcm_fc 2024-01-21T22:07:23.527 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-01-21T22:07:23.527 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/thermal 2024-01-21T22:07:23.527 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/thermal/intel 2024-01-21T22:07:23.527 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/thermal/intel/int340x_thermal 2024-01-21T22:07:23.527 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-01-21T22:07:23.527 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-01-21T22:07:23.528 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-01-21T22:07:23.528 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-01-21T22:07:23.528 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-01-21T22:07:23.528 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-01-21T22:07:23.528 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-01-21T22:07:23.528 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-01-21T22:07:23.528 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-01-21T22:07:23.528 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-01-21T22:07:23.528 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-01-21T22:07:23.528 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-01-21T22:07:23.528 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-01-21T22:07:23.528 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-01-21T22:07:23.528 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-01-21T22:07:23.528 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-01-21T22:07:23.528 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-01-21T22:07:23.529 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-01-21T22:07:23.529 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/tty 2024-01-21T22:07:23.529 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/tty/ipwireless 2024-01-21T22:07:23.529 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-01-21T22:07:23.529 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/tty/n_gsm.ko 2024-01-21T22:07:23.529 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/tty/n_hdlc.ko 2024-01-21T22:07:23.529 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/tty/nozomi.ko 2024-01-21T22:07:23.529 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/tty/serial 2024-01-21T22:07:23.529 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/tty/serial/8250 2024-01-21T22:07:23.529 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-01-21T22:07:23.529 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/tty/serial/arc_uart.ko 2024-01-21T22:07:23.529 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/tty/serial/jsm 2024-01-21T22:07:23.529 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/tty/serial/jsm/jsm.ko 2024-01-21T22:07:23.529 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ufs 2024-01-21T22:07:23.529 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ufs/core 2024-01-21T22:07:23.530 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ufs/core/ufshcd-core.ko 2024-01-21T22:07:23.530 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ufs/host 2024-01-21T22:07:23.530 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-01-21T22:07:23.530 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/uio 2024-01-21T22:07:23.530 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/uio/uio.ko 2024-01-21T22:07:23.530 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/uio/uio_aec.ko 2024-01-21T22:07:23.530 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/uio/uio_cif.ko 2024-01-21T22:07:23.530 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/uio/uio_pci_generic.ko 2024-01-21T22:07:23.530 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/uio/uio_sercos3.ko 2024-01-21T22:07:23.530 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb 2024-01-21T22:07:23.530 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/class 2024-01-21T22:07:23.530 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/class/cdc-acm.ko 2024-01-21T22:07:23.530 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/class/cdc-wdm.ko 2024-01-21T22:07:23.530 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/class/usblp.ko 2024-01-21T22:07:23.530 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/class/usbtmc.ko 2024-01-21T22:07:23.531 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/host 2024-01-21T22:07:23.531 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/host/sl811-hcd.ko 2024-01-21T22:07:23.531 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/image 2024-01-21T22:07:23.531 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/image/mdc800.ko 2024-01-21T22:07:23.531 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/image/microtek.ko 2024-01-21T22:07:23.531 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/misc 2024-01-21T22:07:23.531 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/misc/adutux.ko 2024-01-21T22:07:23.531 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/misc/appledisplay.ko 2024-01-21T22:07:23.531 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/misc/emi26.ko 2024-01-21T22:07:23.531 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/misc/emi62.ko 2024-01-21T22:07:23.531 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/misc/ezusb.ko 2024-01-21T22:07:23.531 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/misc/idmouse.ko 2024-01-21T22:07:23.531 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/misc/iowarrior.ko 2024-01-21T22:07:23.531 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/misc/isight_firmware.ko 2024-01-21T22:07:23.531 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/misc/ldusb.ko 2024-01-21T22:07:23.532 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/misc/legousbtower.ko 2024-01-21T22:07:23.532 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/misc/sisusbvga 2024-01-21T22:07:23.532 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-01-21T22:07:23.532 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/misc/trancevibrator.ko 2024-01-21T22:07:23.532 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/misc/usb3503.ko 2024-01-21T22:07:23.532 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/misc/usblcd.ko 2024-01-21T22:07:23.532 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/misc/usbsevseg.ko 2024-01-21T22:07:23.532 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/misc/uss720.ko 2024-01-21T22:07:23.532 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/misc/yurex.ko 2024-01-21T22:07:23.532 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/phy 2024-01-21T22:07:23.532 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/phy/phy-generic.ko 2024-01-21T22:07:23.532 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial 2024-01-21T22:07:23.532 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/aircable.ko 2024-01-21T22:07:23.532 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/ark3116.ko 2024-01-21T22:07:23.533 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/belkin_sa.ko 2024-01-21T22:07:23.533 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/ch341.ko 2024-01-21T22:07:23.533 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/cp210x.ko 2024-01-21T22:07:23.533 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/cyberjack.ko 2024-01-21T22:07:23.533 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/cypress_m8.ko 2024-01-21T22:07:23.533 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/digi_acceleport.ko 2024-01-21T22:07:23.533 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/empeg.ko 2024-01-21T22:07:23.533 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/ftdi_sio.ko 2024-01-21T22:07:23.533 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/garmin_gps.ko 2024-01-21T22:07:23.533 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/io_edgeport.ko 2024-01-21T22:07:23.533 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/io_ti.ko 2024-01-21T22:07:23.533 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/ipaq.ko 2024-01-21T22:07:23.533 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/ipw.ko 2024-01-21T22:07:23.533 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/ir-usb.ko 2024-01-21T22:07:23.533 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-01-21T22:07:23.534 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/keyspan.ko 2024-01-21T22:07:23.534 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/keyspan_pda.ko 2024-01-21T22:07:23.534 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/kl5kusb105.ko 2024-01-21T22:07:23.534 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/kobil_sct.ko 2024-01-21T22:07:23.534 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/mct_u232.ko 2024-01-21T22:07:23.534 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/mos7720.ko 2024-01-21T22:07:23.534 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/mos7840.ko 2024-01-21T22:07:23.534 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/navman.ko 2024-01-21T22:07:23.534 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/omninet.ko 2024-01-21T22:07:23.534 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/opticon.ko 2024-01-21T22:07:23.534 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/option.ko 2024-01-21T22:07:23.534 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/oti6858.ko 2024-01-21T22:07:23.534 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/pl2303.ko 2024-01-21T22:07:23.534 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/qcaux.ko 2024-01-21T22:07:23.534 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/qcserial.ko 2024-01-21T22:07:23.535 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/quatech2.ko 2024-01-21T22:07:23.535 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/safe_serial.ko 2024-01-21T22:07:23.535 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/sierra.ko 2024-01-21T22:07:23.535 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/spcp8x5.ko 2024-01-21T22:07:23.535 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/ssu100.ko 2024-01-21T22:07:23.535 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/symbolserial.ko 2024-01-21T22:07:23.535 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-01-21T22:07:23.535 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-01-21T22:07:23.535 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/usb_debug.ko 2024-01-21T22:07:23.535 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/usb_wwan.ko 2024-01-21T22:07:23.535 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/visor.ko 2024-01-21T22:07:23.535 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/whiteheat.ko 2024-01-21T22:07:23.535 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/serial/xsens_mt.ko 2024-01-21T22:07:23.535 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/storage 2024-01-21T22:07:23.535 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/storage/uas.ko 2024-01-21T22:07:23.535 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/storage/ums-alauda.ko 2024-01-21T22:07:23.536 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/storage/ums-cypress.ko 2024-01-21T22:07:23.536 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/storage/ums-datafab.ko 2024-01-21T22:07:23.536 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-01-21T22:07:23.536 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/storage/ums-freecom.ko 2024-01-21T22:07:23.536 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/storage/ums-isd200.ko 2024-01-21T22:07:23.536 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-01-21T22:07:23.536 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/storage/ums-karma.ko 2024-01-21T22:07:23.536 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/storage/ums-onetouch.ko 2024-01-21T22:07:23.536 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/storage/ums-realtek.ko 2024-01-21T22:07:23.536 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/storage/ums-sddr09.ko 2024-01-21T22:07:23.536 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/storage/ums-sddr55.ko 2024-01-21T22:07:23.536 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/storage/ums-usbat.ko 2024-01-21T22:07:23.536 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/storage/usb-storage.ko 2024-01-21T22:07:23.536 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/usbip 2024-01-21T22:07:23.537 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/usbip/usbip-core.ko 2024-01-21T22:07:23.537 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/usbip/usbip-host.ko 2024-01-21T22:07:23.537 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-01-21T22:07:23.537 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/vfio 2024-01-21T22:07:23.537 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/vfio/pci 2024-01-21T22:07:23.537 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-01-21T22:07:23.537 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/vfio/pci/vfio-pci.ko 2024-01-21T22:07:23.537 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/vfio/vfio.ko 2024-01-21T22:07:23.537 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-01-21T22:07:23.537 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/vhost 2024-01-21T22:07:23.537 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/vhost/vhost.ko 2024-01-21T22:07:23.537 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/vhost/vhost_iotlb.ko 2024-01-21T22:07:23.537 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/vhost/vhost_net.ko 2024-01-21T22:07:23.537 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/vhost/vhost_scsi.ko 2024-01-21T22:07:23.537 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/video 2024-01-21T22:07:23.537 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/video/backlight 2024-01-21T22:07:23.538 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/video/backlight/apple_bl.ko 2024-01-21T22:07:23.538 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/video/backlight/lcd.ko 2024-01-21T22:07:23.538 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/video/backlight/platform_lcd.ko 2024-01-21T22:07:23.538 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/virtio 2024-01-21T22:07:23.538 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/virtio/virtio.ko 2024-01-21T22:07:23.538 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/virtio/virtio_balloon.ko 2024-01-21T22:07:23.538 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/virtio/virtio_mmio.ko 2024-01-21T22:07:23.538 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/virtio/virtio_pci.ko 2024-01-21T22:07:23.538 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-01-21T22:07:23.538 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-01-21T22:07:23.538 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/virtio/virtio_ring.ko 2024-01-21T22:07:23.538 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/w1 2024-01-21T22:07:23.538 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/w1/masters 2024-01-21T22:07:23.538 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/w1/masters/ds2482.ko 2024-01-21T22:07:23.538 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/w1/masters/ds2490.ko 2024-01-21T22:07:23.539 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/w1/slaves 2024-01-21T22:07:23.539 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-01-21T22:07:23.539 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-01-21T22:07:23.539 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-01-21T22:07:23.539 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-01-21T22:07:23.539 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-01-21T22:07:23.539 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-01-21T22:07:23.539 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-01-21T22:07:23.539 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-01-21T22:07:23.539 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-01-21T22:07:23.539 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/w1/slaves/w1_smem.ko 2024-01-21T22:07:23.539 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/w1/slaves/w1_therm.ko 2024-01-21T22:07:23.539 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/w1/wire.ko 2024-01-21T22:07:23.539 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog 2024-01-21T22:07:23.539 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/alim1535_wdt.ko 2024-01-21T22:07:23.540 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/alim7101_wdt.ko 2024-01-21T22:07:23.540 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/f71808e_wdt.ko 2024-01-21T22:07:23.540 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/hpwdt.ko 2024-01-21T22:07:23.540 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/i6300esb.ko 2024-01-21T22:07:23.540 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-01-21T22:07:23.540 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/iTCO_wdt.ko 2024-01-21T22:07:23.540 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/ib700wdt.ko 2024-01-21T22:07:23.540 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/ibmasr.ko 2024-01-21T22:07:23.540 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-01-21T22:07:23.540 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/it8712f_wdt.ko 2024-01-21T22:07:23.540 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/it87_wdt.ko 2024-01-21T22:07:23.540 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/machzwd.ko 2024-01-21T22:07:23.540 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/nv_tco.ko 2024-01-21T22:07:23.540 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/pcwd_pci.ko 2024-01-21T22:07:23.540 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/pcwd_usb.ko 2024-01-21T22:07:23.541 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-01-21T22:07:23.541 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/sch311x_wdt.ko 2024-01-21T22:07:23.541 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/softdog.ko 2024-01-21T22:07:23.541 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/sp5100_tco.ko 2024-01-21T22:07:23.541 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/via_wdt.ko 2024-01-21T22:07:23.541 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-01-21T22:07:23.541 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/w83877f_wdt.ko 2024-01-21T22:07:23.541 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/w83977f_wdt.ko 2024-01-21T22:07:23.541 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/wdt_pci.ko 2024-01-21T22:07:23.541 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/watchdog/xen_wdt.ko 2024-01-21T22:07:23.541 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/xen 2024-01-21T22:07:23.541 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/xen/xen-acpi-processor.ko 2024-01-21T22:07:23.541 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/xen/xen-evtchn.ko 2024-01-21T22:07:23.541 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/xen/xen-gntalloc.ko 2024-01-21T22:07:23.542 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/xen/xen-gntdev.ko 2024-01-21T22:07:23.542 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/xen/xen-pciback 2024-01-21T22:07:23.542 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-01-21T22:07:23.542 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/xen/xen-privcmd.ko 2024-01-21T22:07:23.542 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/xen/xenfs 2024-01-21T22:07:23.542 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/drivers/xen/xenfs/xenfs.ko 2024-01-21T22:07:23.542 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs 2024-01-21T22:07:23.542 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/9p 2024-01-21T22:07:23.542 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/9p/9p.ko 2024-01-21T22:07:23.542 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/affs 2024-01-21T22:07:23.542 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/affs/affs.ko 2024-01-21T22:07:23.542 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/befs 2024-01-21T22:07:23.542 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/befs/befs.ko 2024-01-21T22:07:23.542 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/binfmt_misc.ko 2024-01-21T22:07:23.542 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/btrfs 2024-01-21T22:07:23.543 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/btrfs/btrfs.ko 2024-01-21T22:07:23.543 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/cachefiles 2024-01-21T22:07:23.543 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/cachefiles/cachefiles.ko 2024-01-21T22:07:23.543 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/ceph 2024-01-21T22:07:23.543 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/ceph/ceph.ko 2024-01-21T22:07:23.543 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/coda 2024-01-21T22:07:23.543 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/coda/coda.ko 2024-01-21T22:07:23.543 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/cramfs 2024-01-21T22:07:23.543 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/cramfs/cramfs.ko 2024-01-21T22:07:23.543 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/dlm 2024-01-21T22:07:23.543 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/dlm/dlm.ko 2024-01-21T22:07:23.543 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/ecryptfs 2024-01-21T22:07:23.543 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/ecryptfs/ecryptfs.ko 2024-01-21T22:07:23.543 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/f2fs 2024-01-21T22:07:23.543 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/f2fs/f2fs.ko 2024-01-21T22:07:23.544 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/fat 2024-01-21T22:07:23.544 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/fat/fat.ko 2024-01-21T22:07:23.544 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/fat/msdos.ko 2024-01-21T22:07:23.544 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/fat/vfat.ko 2024-01-21T22:07:23.544 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/fscache 2024-01-21T22:07:23.544 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/fscache/fscache.ko 2024-01-21T22:07:23.544 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/fuse 2024-01-21T22:07:23.544 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/fuse/cuse.ko 2024-01-21T22:07:23.544 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/fuse/fuse.ko 2024-01-21T22:07:23.544 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/gfs2 2024-01-21T22:07:23.544 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/gfs2/gfs2.ko 2024-01-21T22:07:23.544 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/hfs 2024-01-21T22:07:23.544 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/hfs/hfs.ko 2024-01-21T22:07:23.544 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/hfsplus 2024-01-21T22:07:23.545 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/hfsplus/hfsplus.ko 2024-01-21T22:07:23.545 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/isofs 2024-01-21T22:07:23.545 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/isofs/isofs.ko 2024-01-21T22:07:23.545 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/jfs 2024-01-21T22:07:23.545 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/jfs/jfs.ko 2024-01-21T22:07:23.545 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/lockd 2024-01-21T22:07:23.545 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/lockd/lockd.ko 2024-01-21T22:07:23.545 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/minix 2024-01-21T22:07:23.545 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/minix/minix.ko 2024-01-21T22:07:23.545 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/netfs 2024-01-21T22:07:23.545 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/netfs/netfs.ko 2024-01-21T22:07:23.545 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nfs 2024-01-21T22:07:23.545 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nfs/blocklayout 2024-01-21T22:07:23.545 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-01-21T22:07:23.545 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nfs/filelayout 2024-01-21T22:07:23.546 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-01-21T22:07:23.546 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nfs/flexfilelayout 2024-01-21T22:07:23.546 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-01-21T22:07:23.546 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nfs/nfs.ko 2024-01-21T22:07:23.546 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nfs/nfsv3.ko 2024-01-21T22:07:23.546 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nfs/nfsv4.ko 2024-01-21T22:07:23.546 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nfs_common 2024-01-21T22:07:23.546 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nfs_common/grace.ko 2024-01-21T22:07:23.546 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nfs_common/nfs_acl.ko 2024-01-21T22:07:23.546 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nfsd 2024-01-21T22:07:23.546 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nfsd/nfsd.ko 2024-01-21T22:07:23.546 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nilfs2 2024-01-21T22:07:23.546 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nilfs2/nilfs2.ko 2024-01-21T22:07:23.546 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls 2024-01-21T22:07:23.547 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/mac-celtic.ko 2024-01-21T22:07:23.547 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/mac-centeuro.ko 2024-01-21T22:07:23.547 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/mac-croatian.ko 2024-01-21T22:07:23.547 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/mac-cyrillic.ko 2024-01-21T22:07:23.547 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/mac-gaelic.ko 2024-01-21T22:07:23.547 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/mac-greek.ko 2024-01-21T22:07:23.547 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/mac-iceland.ko 2024-01-21T22:07:23.547 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/mac-inuit.ko 2024-01-21T22:07:23.547 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/mac-roman.ko 2024-01-21T22:07:23.547 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/mac-romanian.ko 2024-01-21T22:07:23.547 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/mac-turkish.ko 2024-01-21T22:07:23.547 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_cp1250.ko 2024-01-21T22:07:23.547 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_cp1251.ko 2024-01-21T22:07:23.547 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_cp1255.ko 2024-01-21T22:07:23.548 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_cp737.ko 2024-01-21T22:07:23.548 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_cp775.ko 2024-01-21T22:07:23.548 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_cp850.ko 2024-01-21T22:07:23.548 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_cp852.ko 2024-01-21T22:07:23.548 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_cp855.ko 2024-01-21T22:07:23.548 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_cp857.ko 2024-01-21T22:07:23.548 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_cp860.ko 2024-01-21T22:07:23.548 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_cp861.ko 2024-01-21T22:07:23.548 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_cp862.ko 2024-01-21T22:07:23.548 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_cp863.ko 2024-01-21T22:07:23.548 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_cp864.ko 2024-01-21T22:07:23.548 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_cp865.ko 2024-01-21T22:07:23.548 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_cp866.ko 2024-01-21T22:07:23.548 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_cp869.ko 2024-01-21T22:07:23.548 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_cp874.ko 2024-01-21T22:07:23.549 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_cp932.ko 2024-01-21T22:07:23.549 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_cp936.ko 2024-01-21T22:07:23.549 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_cp949.ko 2024-01-21T22:07:23.549 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_cp950.ko 2024-01-21T22:07:23.549 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_euc-jp.ko 2024-01-21T22:07:23.549 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_iso8859-1.ko 2024-01-21T22:07:23.549 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_iso8859-13.ko 2024-01-21T22:07:23.549 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_iso8859-14.ko 2024-01-21T22:07:23.549 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_iso8859-15.ko 2024-01-21T22:07:23.549 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_iso8859-2.ko 2024-01-21T22:07:23.549 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_iso8859-3.ko 2024-01-21T22:07:23.549 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_iso8859-4.ko 2024-01-21T22:07:23.549 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_iso8859-5.ko 2024-01-21T22:07:23.549 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_iso8859-6.ko 2024-01-21T22:07:23.549 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_iso8859-7.ko 2024-01-21T22:07:23.550 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_iso8859-9.ko 2024-01-21T22:07:23.550 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_koi8-r.ko 2024-01-21T22:07:23.550 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_koi8-ru.ko 2024-01-21T22:07:23.550 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_koi8-u.ko 2024-01-21T22:07:23.550 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_ucs2_utils.ko 2024-01-21T22:07:23.550 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/nls/nls_utf8.ko 2024-01-21T22:07:23.550 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/ocfs2 2024-01-21T22:07:23.550 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/ocfs2/cluster 2024-01-21T22:07:23.550 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-01-21T22:07:23.550 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/ocfs2/dlm 2024-01-21T22:07:23.550 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-01-21T22:07:23.550 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/ocfs2/dlmfs 2024-01-21T22:07:23.550 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-01-21T22:07:23.550 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/ocfs2/ocfs2.ko 2024-01-21T22:07:23.550 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-01-21T22:07:23.551 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-01-21T22:07:23.551 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-01-21T22:07:23.551 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/overlayfs 2024-01-21T22:07:23.551 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/overlayfs/overlay.ko 2024-01-21T22:07:23.551 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/pstore 2024-01-21T22:07:23.551 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/pstore/ramoops.ko 2024-01-21T22:07:23.551 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/reiserfs 2024-01-21T22:07:23.551 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/reiserfs/reiserfs.ko 2024-01-21T22:07:23.551 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/romfs 2024-01-21T22:07:23.551 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/romfs/romfs.ko 2024-01-21T22:07:23.551 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/smb 2024-01-21T22:07:23.551 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/smb/client 2024-01-21T22:07:23.551 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/smb/client/cifs.ko 2024-01-21T22:07:23.551 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/smb/common 2024-01-21T22:07:23.552 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/smb/common/cifs_arc4.ko 2024-01-21T22:07:23.552 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/smb/common/cifs_md4.ko 2024-01-21T22:07:23.552 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/squashfs 2024-01-21T22:07:23.552 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/squashfs/squashfs.ko 2024-01-21T22:07:23.552 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/sysv 2024-01-21T22:07:23.552 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/sysv/sysv.ko 2024-01-21T22:07:23.552 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/udf 2024-01-21T22:07:23.552 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/udf/udf.ko 2024-01-21T22:07:23.552 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/ufs 2024-01-21T22:07:23.552 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/ufs/ufs.ko 2024-01-21T22:07:23.552 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/xfs 2024-01-21T22:07:23.552 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/fs/xfs/xfs.ko 2024-01-21T22:07:23.552 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/kernel 2024-01-21T22:07:23.552 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/kernel/rcu 2024-01-21T22:07:23.553 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/kernel/rcu/rcutorture.ko 2024-01-21T22:07:23.553 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/kernel/torture.ko 2024-01-21T22:07:23.553 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/kernel/trace 2024-01-21T22:07:23.553 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-01-21T22:07:23.553 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/lib 2024-01-21T22:07:23.553 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/lib/asn1_encoder.ko 2024-01-21T22:07:23.553 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/lib/crc-itu-t.ko 2024-01-21T22:07:23.553 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/lib/crc8.ko 2024-01-21T22:07:23.553 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/lib/crypto 2024-01-21T22:07:23.553 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/lib/crypto/libarc4.ko 2024-01-21T22:07:23.553 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/lib/crypto/libdes.ko 2024-01-21T22:07:23.553 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/lib/libcrc32c.ko 2024-01-21T22:07:23.553 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/lib/lru_cache.ko 2024-01-21T22:07:23.553 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/lib/lz4 2024-01-21T22:07:23.553 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/lib/lz4/lz4_compress.ko 2024-01-21T22:07:23.554 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/lib/lz4/lz4hc_compress.ko 2024-01-21T22:07:23.554 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/lib/math 2024-01-21T22:07:23.554 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/lib/math/cordic.ko 2024-01-21T22:07:23.554 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/lib/raid6 2024-01-21T22:07:23.554 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/lib/raid6/raid6_pq.ko 2024-01-21T22:07:23.554 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/lib/reed_solomon 2024-01-21T22:07:23.554 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/lib/reed_solomon/reed_solomon.ko 2024-01-21T22:07:23.554 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/lib/ts_bm.ko 2024-01-21T22:07:23.554 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/lib/ts_fsm.ko 2024-01-21T22:07:23.554 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/lib/ts_kmp.ko 2024-01-21T22:07:23.554 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/lib/zstd 2024-01-21T22:07:23.554 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/lib/zstd/zstd_compress.ko 2024-01-21T22:07:23.554 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/mm 2024-01-21T22:07:23.554 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/mm/hwpoison-inject.ko 2024-01-21T22:07:23.554 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net 2024-01-21T22:07:23.554 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/6lowpan 2024-01-21T22:07:23.554 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/6lowpan/6lowpan.ko 2024-01-21T22:07:23.555 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/6lowpan/nhc_dest.ko 2024-01-21T22:07:23.555 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/6lowpan/nhc_fragment.ko 2024-01-21T22:07:23.555 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/6lowpan/nhc_hop.ko 2024-01-21T22:07:23.555 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/6lowpan/nhc_ipv6.ko 2024-01-21T22:07:23.555 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/6lowpan/nhc_mobility.ko 2024-01-21T22:07:23.555 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/6lowpan/nhc_routing.ko 2024-01-21T22:07:23.555 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/6lowpan/nhc_udp.ko 2024-01-21T22:07:23.555 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/802 2024-01-21T22:07:23.555 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/802/garp.ko 2024-01-21T22:07:23.555 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/802/mrp.ko 2024-01-21T22:07:23.555 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/802/p8022.ko 2024-01-21T22:07:23.555 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/802/psnap.ko 2024-01-21T22:07:23.555 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/802/stp.ko 2024-01-21T22:07:23.555 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/8021q 2024-01-21T22:07:23.555 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/8021q/8021q.ko 2024-01-21T22:07:23.555 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/9p 2024-01-21T22:07:23.556 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/9p/9pnet.ko 2024-01-21T22:07:23.556 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/9p/9pnet_fd.ko 2024-01-21T22:07:23.556 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/9p/9pnet_rdma.ko 2024-01-21T22:07:23.556 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/9p/9pnet_virtio.ko 2024-01-21T22:07:23.556 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/batman-adv 2024-01-21T22:07:23.556 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/batman-adv/batman-adv.ko 2024-01-21T22:07:23.556 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge 2024-01-21T22:07:23.556 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/br_netfilter.ko 2024-01-21T22:07:23.556 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/bridge.ko 2024-01-21T22:07:23.556 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter 2024-01-21T22:07:23.556 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-01-21T22:07:23.556 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/ebt_among.ko 2024-01-21T22:07:23.556 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/ebt_arp.ko 2024-01-21T22:07:23.556 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-01-21T22:07:23.557 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-01-21T22:07:23.557 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/ebt_ip.ko 2024-01-21T22:07:23.557 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-01-21T22:07:23.557 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/ebt_limit.ko 2024-01-21T22:07:23.557 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/ebt_log.ko 2024-01-21T22:07:23.557 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/ebt_mark.ko 2024-01-21T22:07:23.557 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-01-21T22:07:23.557 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-01-21T22:07:23.557 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-01-21T22:07:23.557 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-01-21T22:07:23.557 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/ebt_snat.ko 2024-01-21T22:07:23.557 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/ebt_stp.ko 2024-01-21T22:07:23.557 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-01-21T22:07:23.557 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-01-21T22:07:23.557 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-01-21T22:07:23.558 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-01-21T22:07:23.558 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/ebtables.ko 2024-01-21T22:07:23.558 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-01-21T22:07:23.558 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-01-21T22:07:23.558 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-01-21T22:07:23.558 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ceph 2024-01-21T22:07:23.558 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ceph/libceph.ko 2024-01-21T22:07:23.558 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/core 2024-01-21T22:07:23.558 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/core/failover.ko 2024-01-21T22:07:23.558 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/core/pktgen.ko 2024-01-21T22:07:23.558 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/dccp 2024-01-21T22:07:23.558 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/dccp/dccp.ko 2024-01-21T22:07:23.558 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/dccp/dccp_diag.ko 2024-01-21T22:07:23.558 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/dccp/dccp_ipv4.ko 2024-01-21T22:07:23.558 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/dccp/dccp_ipv6.ko 2024-01-21T22:07:23.559 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/dns_resolver 2024-01-21T22:07:23.559 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/dns_resolver/dns_resolver.ko 2024-01-21T22:07:23.559 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/dsa 2024-01-21T22:07:23.559 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/dsa/dsa_core.ko 2024-01-21T22:07:23.559 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/dsa/tag_brcm.ko 2024-01-21T22:07:23.559 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/dsa/tag_dsa.ko 2024-01-21T22:07:23.559 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/dsa/tag_none.ko 2024-01-21T22:07:23.559 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/dsa/tag_trailer.ko 2024-01-21T22:07:23.559 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ieee802154 2024-01-21T22:07:23.559 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ieee802154/6lowpan 2024-01-21T22:07:23.559 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-01-21T22:07:23.559 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ieee802154/ieee802154.ko 2024-01-21T22:07:23.559 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ieee802154/ieee802154_socket.ko 2024-01-21T22:07:23.559 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4 2024-01-21T22:07:23.560 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/ah4.ko 2024-01-21T22:07:23.560 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/esp4.ko 2024-01-21T22:07:23.560 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/fou.ko 2024-01-21T22:07:23.560 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/gre.ko 2024-01-21T22:07:23.560 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/inet_diag.ko 2024-01-21T22:07:23.560 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/ip_gre.ko 2024-01-21T22:07:23.560 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/ip_tunnel.ko 2024-01-21T22:07:23.560 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/ip_vti.ko 2024-01-21T22:07:23.560 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/ipcomp.ko 2024-01-21T22:07:23.560 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/ipip.ko 2024-01-21T22:07:23.560 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/netfilter 2024-01-21T22:07:23.560 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/netfilter/arp_tables.ko 2024-01-21T22:07:23.560 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-01-21T22:07:23.560 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-01-21T22:07:23.561 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-01-21T22:07:23.561 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-01-21T22:07:23.561 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-01-21T22:07:23.561 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-01-21T22:07:23.561 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-01-21T22:07:23.561 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-01-21T22:07:23.561 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-01-21T22:07:23.561 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/netfilter/iptable_security.ko 2024-01-21T22:07:23.561 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-01-21T22:07:23.561 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-01-21T22:07:23.561 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-01-21T22:07:23.561 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-01-21T22:07:23.561 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-01-21T22:07:23.561 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-01-21T22:07:23.562 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-01-21T22:07:23.562 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-01-21T22:07:23.562 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-01-21T22:07:23.562 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-01-21T22:07:23.562 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/tcp_bic.ko 2024-01-21T22:07:23.562 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/tcp_dctcp.ko 2024-01-21T22:07:23.562 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/tcp_diag.ko 2024-01-21T22:07:23.562 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/tcp_highspeed.ko 2024-01-21T22:07:23.562 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/tcp_htcp.ko 2024-01-21T22:07:23.562 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/tcp_hybla.ko 2024-01-21T22:07:23.562 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/tcp_illinois.ko 2024-01-21T22:07:23.562 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/tcp_lp.ko 2024-01-21T22:07:23.562 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/tcp_scalable.ko 2024-01-21T22:07:23.562 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/tcp_vegas.ko 2024-01-21T22:07:23.563 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/tcp_veno.ko 2024-01-21T22:07:23.563 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/tcp_westwood.ko 2024-01-21T22:07:23.563 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/tcp_yeah.ko 2024-01-21T22:07:23.563 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/tunnel4.ko 2024-01-21T22:07:23.563 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/udp_diag.ko 2024-01-21T22:07:23.563 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/udp_tunnel.ko 2024-01-21T22:07:23.563 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv4/xfrm4_tunnel.ko 2024-01-21T22:07:23.563 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6 2024-01-21T22:07:23.563 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/ah6.ko 2024-01-21T22:07:23.563 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/esp6.ko 2024-01-21T22:07:23.563 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/fou6.ko 2024-01-21T22:07:23.563 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/ip6_tunnel.ko 2024-01-21T22:07:23.563 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-01-21T22:07:23.563 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/ip6_vti.ko 2024-01-21T22:07:23.563 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/ipcomp6.ko 2024-01-21T22:07:23.564 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter 2024-01-21T22:07:23.564 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-01-21T22:07:23.564 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-01-21T22:07:23.564 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-01-21T22:07:23.564 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-01-21T22:07:23.564 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-01-21T22:07:23.564 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-01-21T22:07:23.564 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-01-21T22:07:23.564 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-01-21T22:07:23.564 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-01-21T22:07:23.564 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-01-21T22:07:23.564 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-01-21T22:07:23.564 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-01-21T22:07:23.564 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-01-21T22:07:23.565 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-01-21T22:07:23.565 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-01-21T22:07:23.565 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-01-21T22:07:23.565 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-01-21T22:07:23.565 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-01-21T22:07:23.565 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-01-21T22:07:23.565 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-01-21T22:07:23.565 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-01-21T22:07:23.565 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-01-21T22:07:23.565 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-01-21T22:07:23.565 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-01-21T22:07:23.565 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/sit.ko 2024-01-21T22:07:23.565 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/tunnel6.ko 2024-01-21T22:07:23.565 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/ipv6/xfrm6_tunnel.ko 2024-01-21T22:07:23.566 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/key 2024-01-21T22:07:23.566 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/key/af_key.ko 2024-01-21T22:07:23.566 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/l2tp 2024-01-21T22:07:23.566 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/l2tp/l2tp_core.ko 2024-01-21T22:07:23.566 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/l2tp/l2tp_debugfs.ko 2024-01-21T22:07:23.566 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/l2tp/l2tp_eth.ko 2024-01-21T22:07:23.566 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/l2tp/l2tp_ip.ko 2024-01-21T22:07:23.566 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/l2tp/l2tp_ip6.ko 2024-01-21T22:07:23.566 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/l2tp/l2tp_netlink.ko 2024-01-21T22:07:23.566 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/l2tp/l2tp_ppp.ko 2024-01-21T22:07:23.566 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/llc 2024-01-21T22:07:23.566 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/llc/llc.ko 2024-01-21T22:07:23.566 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/mac802154 2024-01-21T22:07:23.566 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/mac802154/mac802154.ko 2024-01-21T22:07:23.566 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/mpls 2024-01-21T22:07:23.566 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/mpls/mpls_gso.ko 2024-01-21T22:07:23.567 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter 2024-01-21T22:07:23.567 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipset 2024-01-21T22:07:23.567 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipset/ip_set.ko 2024-01-21T22:07:23.567 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-01-21T22:07:23.567 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-01-21T22:07:23.567 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-01-21T22:07:23.567 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-01-21T22:07:23.567 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-01-21T22:07:23.567 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-01-21T22:07:23.567 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-01-21T22:07:23.567 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-01-21T22:07:23.567 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-01-21T22:07:23.567 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-01-21T22:07:23.567 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-01-21T22:07:23.567 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-01-21T22:07:23.568 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-01-21T22:07:23.568 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-01-21T22:07:23.568 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-01-21T22:07:23.568 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipvs 2024-01-21T22:07:23.568 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipvs/ip_vs.ko 2024-01-21T22:07:23.568 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-01-21T22:07:23.568 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-01-21T22:07:23.568 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-01-21T22:07:23.568 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-01-21T22:07:23.568 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-01-21T22:07:23.568 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-01-21T22:07:23.568 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-01-21T22:07:23.568 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-01-21T22:07:23.568 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-01-21T22:07:23.569 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-01-21T22:07:23.569 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-01-21T22:07:23.569 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-01-21T22:07:23.569 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-01-21T22:07:23.569 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_conncount.ko 2024-01-21T22:07:23.569 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_conntrack.ko 2024-01-21T22:07:23.569 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-01-21T22:07:23.569 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-01-21T22:07:23.569 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-01-21T22:07:23.569 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_conntrack_h323.ko 2024-01-21T22:07:23.569 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_conntrack_irc.ko 2024-01-21T22:07:23.569 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-01-21T22:07:23.569 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-01-21T22:07:23.569 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-01-21T22:07:23.569 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_conntrack_sane.ko 2024-01-21T22:07:23.569 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_conntrack_sip.ko 2024-01-21T22:07:23.570 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-01-21T22:07:23.570 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-01-21T22:07:23.570 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_dup_netdev.ko 2024-01-21T22:07:23.570 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_flow_table.ko 2024-01-21T22:07:23.570 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_flow_table_inet.ko 2024-01-21T22:07:23.570 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_log_syslog.ko 2024-01-21T22:07:23.570 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_nat.ko 2024-01-21T22:07:23.570 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_nat_amanda.ko 2024-01-21T22:07:23.570 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_nat_ftp.ko 2024-01-21T22:07:23.570 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_nat_irc.ko 2024-01-21T22:07:23.570 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_nat_sip.ko 2024-01-21T22:07:23.570 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_nat_tftp.ko 2024-01-21T22:07:23.570 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_synproxy_core.ko 2024-01-21T22:07:23.570 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nf_tables.ko 2024-01-21T22:07:23.570 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nfnetlink.ko 2024-01-21T22:07:23.570 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nfnetlink_acct.ko 2024-01-21T22:07:23.571 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nfnetlink_log.ko 2024-01-21T22:07:23.571 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nfnetlink_osf.ko 2024-01-21T22:07:23.571 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nfnetlink_queue.ko 2024-01-21T22:07:23.571 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nft_chain_nat.ko 2024-01-21T22:07:23.571 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nft_compat.ko 2024-01-21T22:07:23.571 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nft_ct.ko 2024-01-21T22:07:23.571 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nft_dup_netdev.ko 2024-01-21T22:07:23.571 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nft_fib.ko 2024-01-21T22:07:23.571 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nft_fwd_netdev.ko 2024-01-21T22:07:23.571 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nft_hash.ko 2024-01-21T22:07:23.571 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nft_limit.ko 2024-01-21T22:07:23.571 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nft_log.ko 2024-01-21T22:07:23.571 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nft_masq.ko 2024-01-21T22:07:23.571 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nft_nat.ko 2024-01-21T22:07:23.572 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nft_numgen.ko 2024-01-21T22:07:23.572 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nft_queue.ko 2024-01-21T22:07:23.572 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nft_quota.ko 2024-01-21T22:07:23.572 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nft_redir.ko 2024-01-21T22:07:23.572 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nft_reject.ko 2024-01-21T22:07:23.572 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nft_reject_inet.ko 2024-01-21T22:07:23.572 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nft_socket.ko 2024-01-21T22:07:23.572 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/nft_tunnel.ko 2024-01-21T22:07:23.572 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_AUDIT.ko 2024-01-21T22:07:23.572 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_CHECKSUM.ko 2024-01-21T22:07:23.572 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_CLASSIFY.ko 2024-01-21T22:07:23.572 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-01-21T22:07:23.572 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_CT.ko 2024-01-21T22:07:23.572 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_DSCP.ko 2024-01-21T22:07:23.573 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_HL.ko 2024-01-21T22:07:23.573 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_HMARK.ko 2024-01-21T22:07:23.573 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_IDLETIMER.ko 2024-01-21T22:07:23.573 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_LED.ko 2024-01-21T22:07:23.573 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_LOG.ko 2024-01-21T22:07:23.573 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_MASQUERADE.ko 2024-01-21T22:07:23.573 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_NETMAP.ko 2024-01-21T22:07:23.573 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_NFLOG.ko 2024-01-21T22:07:23.573 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_NFQUEUE.ko 2024-01-21T22:07:23.573 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_RATEEST.ko 2024-01-21T22:07:23.573 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_REDIRECT.ko 2024-01-21T22:07:23.573 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_SECMARK.ko 2024-01-21T22:07:23.573 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_TCPMSS.ko 2024-01-21T22:07:23.573 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-01-21T22:07:23.574 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_TEE.ko 2024-01-21T22:07:23.574 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_TPROXY.ko 2024-01-21T22:07:23.574 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_TRACE.ko 2024-01-21T22:07:23.574 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_addrtype.ko 2024-01-21T22:07:23.574 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_bpf.ko 2024-01-21T22:07:23.574 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_cgroup.ko 2024-01-21T22:07:23.574 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_cluster.ko 2024-01-21T22:07:23.574 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_comment.ko 2024-01-21T22:07:23.574 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_connbytes.ko 2024-01-21T22:07:23.574 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_connlabel.ko 2024-01-21T22:07:23.574 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_connlimit.ko 2024-01-21T22:07:23.574 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_connmark.ko 2024-01-21T22:07:23.574 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_conntrack.ko 2024-01-21T22:07:23.574 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_cpu.ko 2024-01-21T22:07:23.574 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_dccp.ko 2024-01-21T22:07:23.574 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_devgroup.ko 2024-01-21T22:07:23.575 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_dscp.ko 2024-01-21T22:07:23.575 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_ecn.ko 2024-01-21T22:07:23.575 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_esp.ko 2024-01-21T22:07:23.575 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_hashlimit.ko 2024-01-21T22:07:23.575 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_helper.ko 2024-01-21T22:07:23.575 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_hl.ko 2024-01-21T22:07:23.575 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_ipcomp.ko 2024-01-21T22:07:23.575 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_iprange.ko 2024-01-21T22:07:23.575 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_ipvs.ko 2024-01-21T22:07:23.575 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_l2tp.ko 2024-01-21T22:07:23.575 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_length.ko 2024-01-21T22:07:23.575 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_limit.ko 2024-01-21T22:07:23.575 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_mac.ko 2024-01-21T22:07:23.575 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_mark.ko 2024-01-21T22:07:23.575 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_multiport.ko 2024-01-21T22:07:23.576 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_nat.ko 2024-01-21T22:07:23.576 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_nfacct.ko 2024-01-21T22:07:23.576 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_osf.ko 2024-01-21T22:07:23.576 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_owner.ko 2024-01-21T22:07:23.576 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_physdev.ko 2024-01-21T22:07:23.576 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_pkttype.ko 2024-01-21T22:07:23.576 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_policy.ko 2024-01-21T22:07:23.576 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_quota.ko 2024-01-21T22:07:23.576 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_rateest.ko 2024-01-21T22:07:23.576 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_realm.ko 2024-01-21T22:07:23.576 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_recent.ko 2024-01-21T22:07:23.576 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_sctp.ko 2024-01-21T22:07:23.576 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_set.ko 2024-01-21T22:07:23.576 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_socket.ko 2024-01-21T22:07:23.576 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_state.ko 2024-01-21T22:07:23.577 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_statistic.ko 2024-01-21T22:07:23.577 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_string.ko 2024-01-21T22:07:23.577 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_tcpmss.ko 2024-01-21T22:07:23.577 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_time.ko 2024-01-21T22:07:23.577 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netfilter/xt_u32.ko 2024-01-21T22:07:23.577 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netlink 2024-01-21T22:07:23.577 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/netlink/netlink_diag.ko 2024-01-21T22:07:23.577 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/nsh 2024-01-21T22:07:23.577 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/nsh/nsh.ko 2024-01-21T22:07:23.577 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/openvswitch 2024-01-21T22:07:23.577 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/openvswitch/openvswitch.ko 2024-01-21T22:07:23.577 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/openvswitch/vport-geneve.ko 2024-01-21T22:07:23.577 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/openvswitch/vport-gre.ko 2024-01-21T22:07:23.577 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/openvswitch/vport-vxlan.ko 2024-01-21T22:07:23.577 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/packet 2024-01-21T22:07:23.578 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/packet/af_packet_diag.ko 2024-01-21T22:07:23.578 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/rds 2024-01-21T22:07:23.578 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/rds/rds.ko 2024-01-21T22:07:23.578 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/rds/rds_rdma.ko 2024-01-21T22:07:23.578 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/rds/rds_tcp.ko 2024-01-21T22:07:23.578 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched 2024-01-21T22:07:23.578 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/act_csum.ko 2024-01-21T22:07:23.578 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/act_gact.ko 2024-01-21T22:07:23.578 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/act_ipt.ko 2024-01-21T22:07:23.578 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/act_mirred.ko 2024-01-21T22:07:23.578 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/act_nat.ko 2024-01-21T22:07:23.578 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/act_pedit.ko 2024-01-21T22:07:23.578 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/act_police.ko 2024-01-21T22:07:23.578 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/act_simple.ko 2024-01-21T22:07:23.578 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/act_skbedit.ko 2024-01-21T22:07:23.579 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/act_vlan.ko 2024-01-21T22:07:23.579 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/cls_basic.ko 2024-01-21T22:07:23.579 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/cls_bpf.ko 2024-01-21T22:07:23.579 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/cls_flow.ko 2024-01-21T22:07:23.579 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/cls_fw.ko 2024-01-21T22:07:23.579 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/cls_route.ko 2024-01-21T22:07:23.579 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/cls_u32.ko 2024-01-21T22:07:23.579 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/em_cmp.ko 2024-01-21T22:07:23.579 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/em_ipset.ko 2024-01-21T22:07:23.579 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/em_meta.ko 2024-01-21T22:07:23.579 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/em_nbyte.ko 2024-01-21T22:07:23.579 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/em_text.ko 2024-01-21T22:07:23.579 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/em_u32.ko 2024-01-21T22:07:23.579 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/sch_choke.ko 2024-01-21T22:07:23.579 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/sch_codel.ko 2024-01-21T22:07:23.580 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/sch_drr.ko 2024-01-21T22:07:23.580 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/sch_fq.ko 2024-01-21T22:07:23.580 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/sch_gred.ko 2024-01-21T22:07:23.580 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/sch_hfsc.ko 2024-01-21T22:07:23.580 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/sch_hhf.ko 2024-01-21T22:07:23.580 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/sch_htb.ko 2024-01-21T22:07:23.580 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/sch_ingress.ko 2024-01-21T22:07:23.580 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/sch_mqprio.ko 2024-01-21T22:07:23.580 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/sch_mqprio_lib.ko 2024-01-21T22:07:23.580 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/sch_multiq.ko 2024-01-21T22:07:23.580 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/sch_netem.ko 2024-01-21T22:07:23.580 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/sch_pie.ko 2024-01-21T22:07:23.580 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/sch_plug.ko 2024-01-21T22:07:23.580 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/sch_prio.ko 2024-01-21T22:07:23.580 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/sch_qfq.ko 2024-01-21T22:07:23.581 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/sch_red.ko 2024-01-21T22:07:23.581 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/sch_sfb.ko 2024-01-21T22:07:23.581 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/sch_sfq.ko 2024-01-21T22:07:23.581 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/sch_tbf.ko 2024-01-21T22:07:23.581 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sched/sch_teql.ko 2024-01-21T22:07:23.581 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sctp 2024-01-21T22:07:23.581 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sctp/sctp.ko 2024-01-21T22:07:23.581 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sctp/sctp_diag.ko 2024-01-21T22:07:23.581 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sunrpc 2024-01-21T22:07:23.581 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sunrpc/auth_gss 2024-01-21T22:07:23.581 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-01-21T22:07:23.581 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-01-21T22:07:23.581 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sunrpc/sunrpc.ko 2024-01-21T22:07:23.581 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sunrpc/xprtrdma 2024-01-21T22:07:23.581 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-01-21T22:07:23.582 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/tipc 2024-01-21T22:07:23.582 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/tipc/diag.ko 2024-01-21T22:07:23.582 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/tipc/tipc.ko 2024-01-21T22:07:23.582 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/unix 2024-01-21T22:07:23.582 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/unix/unix_diag.ko 2024-01-21T22:07:23.582 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/vmw_vsock 2024-01-21T22:07:23.582 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-01-21T22:07:23.582 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-01-21T22:07:23.582 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/vmw_vsock/vsock.ko 2024-01-21T22:07:23.582 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/vmw_vsock/vsock_diag.ko 2024-01-21T22:07:23.582 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/vmw_vsock/vsock_loopback.ko 2024-01-21T22:07:23.582 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/xfrm 2024-01-21T22:07:23.582 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/net/xfrm/xfrm_ipcomp.ko 2024-01-21T22:07:23.582 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/security 2024-01-21T22:07:23.583 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/security/keys 2024-01-21T22:07:23.583 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/security/keys/encrypted-keys 2024-01-21T22:07:23.583 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-01-21T22:07:23.583 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/security/keys/trusted-keys 2024-01-21T22:07:23.583 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/security/keys/trusted-keys/trusted.ko 2024-01-21T22:07:23.583 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/virt 2024-01-21T22:07:23.583 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/virt/lib 2024-01-21T22:07:23.583 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/kernel/virt/lib/irqbypass.ko 2024-01-21T22:07:23.583 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/modules.alias 2024-01-21T22:07:23.583 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/modules.alias.bin 2024-01-21T22:07:23.583 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/modules.builtin 2024-01-21T22:07:23.583 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/modules.builtin.alias.bin 2024-01-21T22:07:23.583 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/modules.builtin.bin 2024-01-21T22:07:23.583 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/modules.builtin.modinfo 2024-01-21T22:07:23.583 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/modules.dep 2024-01-21T22:07:23.584 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/modules.dep.bin 2024-01-21T22:07:23.584 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/modules.devname 2024-01-21T22:07:23.584 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/modules.order 2024-01-21T22:07:23.584 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/modules.softdep 2024-01-21T22:07:23.584 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/modules.symbols 2024-01-21T22:07:23.584 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.7.0-g06652275eb93/modules.symbols.bin 2024-01-21T22:07:23.585 DEBUG:teuthology.task.kernel:get_image_version: 6.7.0-g06652275eb93 2024-01-21T22:07:23.585 DEBUG:teuthology.orchestra.run.smithi138:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-01-21T22:07:23.607 INFO:teuthology.orchestra.run.smithi138.stdout:postinstall scriptlet (using /bin/sh): 2024-01-21T22:07:23.608 INFO:teuthology.orchestra.run.smithi138.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.7.0-g06652275eb93 -a -r /boot/System.map-6.7.0-g06652275eb93 ]; then 2024-01-21T22:07:23.608 INFO:teuthology.orchestra.run.smithi138.stdout:cp /boot/vmlinuz-6.7.0-g06652275eb93 /boot/.vmlinuz-6.7.0-g06652275eb93-rpm 2024-01-21T22:07:23.608 INFO:teuthology.orchestra.run.smithi138.stdout:cp /boot/System.map-6.7.0-g06652275eb93 /boot/.System.map-6.7.0-g06652275eb93-rpm 2024-01-21T22:07:23.608 INFO:teuthology.orchestra.run.smithi138.stdout:rm -f /boot/vmlinuz-6.7.0-g06652275eb93 /boot/System.map-6.7.0-g06652275eb93 2024-01-21T22:07:23.608 INFO:teuthology.orchestra.run.smithi138.stdout:/sbin/installkernel 6.7.0-g06652275eb93 /boot/.vmlinuz-6.7.0-g06652275eb93-rpm /boot/.System.map-6.7.0-g06652275eb93-rpm 2024-01-21T22:07:23.608 INFO:teuthology.orchestra.run.smithi138.stdout:rm -f /boot/.vmlinuz-6.7.0-g06652275eb93-rpm /boot/.System.map-6.7.0-g06652275eb93-rpm 2024-01-21T22:07:23.608 INFO:teuthology.orchestra.run.smithi138.stdout:fi 2024-01-21T22:07:23.608 INFO:teuthology.orchestra.run.smithi138.stdout:preuninstall scriptlet (using /bin/sh): 2024-01-21T22:07:23.608 INFO:teuthology.orchestra.run.smithi138.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-01-21T22:07:23.608 INFO:teuthology.orchestra.run.smithi138.stdout:new-kernel-pkg --remove 6.7.0-g06652275eb93 --rminitrd --initrdfile=/boot/initramfs-6.7.0-g06652275eb93.img 2024-01-21T22:07:23.608 INFO:teuthology.orchestra.run.smithi138.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-01-21T22:07:23.608 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-install remove 6.7.0-g06652275eb93 2024-01-21T22:07:23.608 INFO:teuthology.orchestra.run.smithi138.stdout:fi 2024-01-21T22:07:23.608 INFO:teuthology.orchestra.run.smithi138.stdout:postuninstall scriptlet (using /bin/sh): 2024-01-21T22:07:23.608 INFO:teuthology.orchestra.run.smithi138.stdout:if [ -x /sbin/update-bootloader ]; then 2024-01-21T22:07:23.609 INFO:teuthology.orchestra.run.smithi138.stdout:/sbin/update-bootloader --remove 6.7.0-g06652275eb93 2024-01-21T22:07:23.609 INFO:teuthology.orchestra.run.smithi138.stdout:fi 2024-01-21T22:07:23.609 DEBUG:teuthology.orchestra.run.smithi138:> sudo rpm -qi grub2-tools 2024-01-21T22:07:23.685 INFO:teuthology.orchestra.run.smithi138.stdout:Name : grub2-tools 2024-01-21T22:07:23.685 INFO:teuthology.orchestra.run.smithi138.stdout:Epoch : 1 2024-01-21T22:07:23.685 INFO:teuthology.orchestra.run.smithi138.stdout:Version : 2.06 2024-01-21T22:07:23.685 INFO:teuthology.orchestra.run.smithi138.stdout:Release : 68.el9 2024-01-21T22:07:23.686 INFO:teuthology.orchestra.run.smithi138.stdout:Architecture: x86_64 2024-01-21T22:07:23.686 INFO:teuthology.orchestra.run.smithi138.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-01-21T22:07:23.686 INFO:teuthology.orchestra.run.smithi138.stdout:Group : Unspecified 2024-01-21T22:07:23.686 INFO:teuthology.orchestra.run.smithi138.stdout:Size : 8264254 2024-01-21T22:07:23.686 INFO:teuthology.orchestra.run.smithi138.stdout:License : GPLv3+ 2024-01-21T22:07:23.686 INFO:teuthology.orchestra.run.smithi138.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-01-21T22:07:23.686 INFO:teuthology.orchestra.run.smithi138.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-01-21T22:07:23.686 INFO:teuthology.orchestra.run.smithi138.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-01-21T22:07:23.686 INFO:teuthology.orchestra.run.smithi138.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-01-21T22:07:23.686 INFO:teuthology.orchestra.run.smithi138.stdout:Packager : builder@centos.org 2024-01-21T22:07:23.686 INFO:teuthology.orchestra.run.smithi138.stdout:Vendor : CentOS 2024-01-21T22:07:23.686 INFO:teuthology.orchestra.run.smithi138.stdout:URL : http://www.gnu.org/software/grub/ 2024-01-21T22:07:23.686 INFO:teuthology.orchestra.run.smithi138.stdout:Summary : Support tools for GRUB. 2024-01-21T22:07:23.686 INFO:teuthology.orchestra.run.smithi138.stdout:Description : 2024-01-21T22:07:23.687 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:07:23.687 INFO:teuthology.orchestra.run.smithi138.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-01-21T22:07:23.687 INFO:teuthology.orchestra.run.smithi138.stdout:customizable bootloader with modular architecture. It supports a rich 2024-01-21T22:07:23.687 INFO:teuthology.orchestra.run.smithi138.stdout:variety of kernel formats, file systems, computer architectures and 2024-01-21T22:07:23.687 INFO:teuthology.orchestra.run.smithi138.stdout:hardware devices. 2024-01-21T22:07:23.687 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:07:23.687 INFO:teuthology.orchestra.run.smithi138.stdout:This subpackage provides tools for support of all platforms. 2024-01-21T22:07:23.688 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-01-21T22:07:23.688 INFO:teuthology.task.kernel:Updating grub on smithi138 to boot 6.7.0-g06652275eb93 2024-01-21T22:07:23.688 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-01-21T22:07:24.276 INFO:teuthology.orchestra.run.smithi138.stderr:Generating grub configuration file ... 2024-01-21T22:07:25.734 INFO:teuthology.orchestra.run.smithi138.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-01-21T22:07:25.774 INFO:teuthology.orchestra.run.smithi138.stderr:done 2024-01-21T22:07:25.776 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-01-21T22:07:25.791 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.k0UDWVa6Lm 2024-01-21T22:07:25.791 DEBUG:teuthology.orchestra.run.smithi138:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.k0UDWVa6Lm 2024-01-21T22:07:25.860 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.k0UDWVa6Lm 2024-01-21T22:07:25.996 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.k0UDWVa6Lm is 6KB 2024-01-21T22:07:26.045 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.k0UDWVa6Lm 2024-01-21T22:07:26.059 DEBUG:teuthology.orchestra.run.smithi138:> sudo /bin/ls /boot/loader/entries || true 2024-01-21T22:07:26.124 INFO:teuthology.orchestra.run.smithi138.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-01-21T22:07:26.124 INFO:teuthology.orchestra.run.smithi138.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-01-21T22:07:26.125 INFO:teuthology.orchestra.run.smithi138.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-01-21T22:07:26.125 INFO:teuthology.orchestra.run.smithi138.stdout:15949e3c88704300882482eddfd7baae-6.7.0-g06652275eb93.conf 2024-01-21T22:07:26.126 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.7.0-g06652275eb93 2024-01-21T22:07:26.272 DEBUG:teuthology.orchestra.run.smithi138:> sudo shutdown -r now 2024-01-21T22:07:26.356 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2024-01-21T22:07:56.275 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-01-21T22:07:56.275 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-01-21T22:07:56.276 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-21T22:08:56.279 DEBUG:teuthology.orchestra.remote:timed out 2024-01-21T22:09:05.280 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-01-21T22:09:05.281 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-21T22:09:08.348 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-01-21T22:09:20.349 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-01-21T22:09:20.350 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-21T22:09:20.790 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-01-21T22:09:21.380 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-01-21T22:09:21.380 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.7.0-g06652275eb93"... 2024-01-21T22:09:21.380 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-01-21T22:09:21.415 INFO:teuthology.orchestra.run.smithi138.stdout:6.7.0-g06652275eb93 2024-01-21T22:09:21.415 DEBUG:teuthology.task.kernel:current kernel version is 6.7.0-g06652275eb93 vs 6.7.0-g06652275eb93 2024-01-21T22:09:21.415 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-01-21T22:09:21.416 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-01-21T22:09:22.419 DEBUG:teuthology.parallel:result is None 2024-01-21T22:09:22.419 INFO:teuthology.run_tasks:Running task internal.base... 2024-01-21T22:09:22.429 INFO:teuthology.task.internal:Creating test directory... 2024-01-21T22:09:22.429 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-21T22:09:22.437 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-01-21T22:09:22.478 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-01-21T22:09:22.484 INFO:teuthology.run_tasks:Running task internal.archive... 2024-01-21T22:09:22.499 INFO:teuthology.task.internal:Creating archive directory... 2024-01-21T22:09:22.499 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-21T22:09:22.523 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-01-21T22:09:22.579 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-01-21T22:09:22.588 INFO:teuthology.task.internal:Enabling coredump saving... 2024-01-21T22:09:22.588 DEBUG:teuthology.orchestra.run.smithi152:> test -f /run/.containerenv -o -f /.dockerenv 2024-01-21T22:09:22.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-21T22:09:22.606 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-21T22:09:22.613 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-01-21T22:09:22.710 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-21T22:09:22.719 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-21T22:09:22.729 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-21T22:09:22.792 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-01-21T22:09:22.794 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-01-21T22:09:22.803 INFO:teuthology.task.internal:Configuring sudo... 2024-01-21T22:09:22.803 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-21T22:09:22.807 DEBUG:teuthology.orchestra.run.smithi152:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-01-21T22:09:22.871 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-01-21T22:09:22.881 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-01-21T22:09:22.882 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-21T22:09:22.886 DEBUG:teuthology.orchestra.run.smithi152:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-01-21T22:09:22.922 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-21T22:09:23.038 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-21T22:09:23.140 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-21T22:09:23.218 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-21T22:09:23.304 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:09:23.304 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-21T22:09:23.395 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-21T22:09:23.442 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-01-21T22:09:23.498 DEBUG:teuthology.orchestra.run.smithi152:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-21T22:09:23.578 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-01-21T22:09:23.627 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-21T22:09:23.627 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-01-21T22:09:23.696 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-01-21T22:09:23.704 DEBUG:teuthology.orchestra.run.smithi152:> sudo service rsyslog restart 2024-01-21T22:09:23.785 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-21T22:09:23.846 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-21T22:09:24.169 INFO:teuthology.run_tasks:Running task internal.timer... 2024-01-21T22:09:24.179 INFO:teuthology.task.internal:Starting timer... 2024-01-21T22:09:24.180 INFO:teuthology.run_tasks:Running task pcp... 2024-01-21T22:09:24.200 INFO:teuthology.run_tasks:Running task selinux... 2024-01-21T22:09:24.216 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2024-01-21T22:09:24.335 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2024-01-21T22:09:24.351 DEBUG:teuthology.orchestra.run.smithi152:> sudo service auditd rotate 2024-01-21T22:09:24.396 INFO:teuthology.orchestra.run.smithi152.stdout:Rotating logs: 2024-01-21T22:09:24.398 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-01-21T22:09:24.398 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2024-01-21T22:09:24.447 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2024-01-21T22:09:24.452 DEBUG:teuthology.orchestra.run.smithi152:> /usr/sbin/getenforce 2024-01-21T22:09:24.485 INFO:teuthology.orchestra.run.smithi152.stdout:Permissive 2024-01-21T22:09:24.485 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi138.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi152.front.sepia.ceph.com': 'permissive'} 2024-01-21T22:09:24.485 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-21T22:09:24.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-21T22:09:24.631 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-21T22:09:24.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-21T22:09:24.665 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-01-21T22:09:24.665 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2024-01-21T22:09:24.804 DEBUG:teuthology.orchestra.run.smithi152:> sudo /usr/sbin/setenforce permissive 2024-01-21T22:09:24.837 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-01-21T22:09:24.848 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-01-21T22:09:24.871 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-01-21T22:09:24.872 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi138.front.sepia.ceph.com,smithi152.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-01-21T22:14:58.931 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi138.front.sepia.ceph.com'), Remote(name='ubuntu@smithi152.front.sepia.ceph.com')] 2024-01-21T22:14:58.932 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-01-21T22:14:58.933 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-21T22:14:59.068 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-01-21T22:14:59.212 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-01-21T22:14:59.213 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-01-21T22:14:59.213 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi152.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-01-21T22:14:59.301 DEBUG:teuthology.orchestra.run.smithi152:> true 2024-01-21T22:14:59.370 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi152.front.sepia.ceph.com' 2024-01-21T22:14:59.370 INFO:teuthology.run_tasks:Running task clock... 2024-01-21T22:14:59.380 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-01-21T22:14:59.380 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-21T22:14:59.381 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-21T22:14:59.386 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-01-21T22:14:59.387 DEBUG:teuthology.orchestra.run.smithi152:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-21T22:14:59.441 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-21T22:14:59.457 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-01-21T22:14:59.457 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-21T22:14:59.486 INFO:teuthology.orchestra.run.smithi152.stderr:sudo: ntpd: command not found 2024-01-21T22:14:59.500 INFO:teuthology.orchestra.run.smithi152.stdout:506 Cannot talk to daemon 2024-01-21T22:14:59.502 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-01-21T22:14:59.516 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-21T22:14:59.532 INFO:teuthology.orchestra.run.smithi152.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-21T22:14:59.576 INFO:teuthology.orchestra.run.smithi152.stderr:bash: line 1: ntpq: command not found 2024-01-21T22:14:59.579 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-21T22:14:59.579 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2024-01-21T22:14:59.580 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-21T22:14:59.580 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-21T22:14:59.580 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-21T22:14:59.580 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-21T22:14:59.589 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2024-01-21T22:14:59.627 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2024-01-21T22:14:59.670 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-01-21T22:14:59.718 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-01-21T22:14:59.833 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-01-21T22:14:59.841 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-21T22:14:59.841 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-01-21T22:14:59.841 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-21T22:14:59.841 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-21T22:14:59.841 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-21T22:14:59.841 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-01-21T22:14:59.842 INFO:teuthology.run_tasks:Running task install... 2024-01-21T22:14:59.851 DEBUG:teuthology.task.install:project ceph 2024-01-21T22:14:59.851 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': 'eb1b049f9003ac37e0a3ab33496ad1924d301254'}} 2024-01-21T22:14:59.851 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'debuginfo': True, 'flavor': 'default', 'sha1': 'eb1b049f9003ac37e0a3ab33496ad1924d301254'} 2024-01-21T22:14:59.851 INFO:teuthology.task.install:Using flavor: default 2024-01-21T22:14:59.857 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse', 'ceph-common-dbg', 'ceph-fuse-dbg', 'ceph-mds-dbg', 'ceph-mgr-dbg', 'ceph-mon-dbg', 'ceph-osd-dbg', 'ceph-test-dbg', 'libcephfs2-dbg', 'librados2-dbg', 'libradosstriper1-dbg', 'librbd1-dbg', 'librgw2-dbg', 'radosgw-dbg', 'rbd-fuse-dbg', 'rbd-mirror-dbg', 'rbd-nbd-dbg'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd', 'ceph-base-debuginfo', 'ceph-common-debuginfo', 'ceph-immutable-object-cache-debuginfo', 'ceph-radosgw-debuginfo', 'ceph-test-debuginfo', 'ceph-base-debuginfo', 'ceph-mgr-debuginfo', 'ceph-mds-debuginfo', 'ceph-mon-debuginfo', 'ceph-osd-debuginfo', 'ceph-fuse-debuginfo', 'librados-devel-debuginfo', 'libcephfs2-debuginfo', 'librados2-debuginfo', 'librbd1-debuginfo', 'python3-cephfs-debuginfo', 'python3-rados-debuginfo', 'python3-rbd-debuginfo', 'python3-rgw-debuginfo', 'rbd-fuse-debuginfo', 'rbd-mirror-debuginfo', 'rbd-nbd-debuginfo']} 2024-01-21T22:14:59.857 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-01-21T22:14:59.857 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'eb1b049f9003ac37e0a3ab33496ad1924d301254', 'tag': None, 'wait_for_package': False} 2024-01-21T22:14:59.858 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=eb1b049f9003ac37e0a3ab33496ad1924d301254 2024-01-21T22:14:59.860 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'eb1b049f9003ac37e0a3ab33496ad1924d301254', 'tag': None, 'wait_for_package': False} 2024-01-21T22:14:59.860 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=eb1b049f9003ac37e0a3ab33496ad1924d301254 2024-01-21T22:15:00.004 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240119.075157/eb1b049f9003ac37e0a3ab33496ad1924d301254/centos/9/flavors/default/ 2024-01-21T22:15:00.004 INFO:teuthology.task.install.rpm:Package version is 19.0.0-811.geb1b049f 2024-01-21T22:15:00.097 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240119.075157/eb1b049f9003ac37e0a3ab33496ad1924d301254/centos/9/flavors/default/ 2024-01-21T22:15:00.097 INFO:teuthology.task.install.rpm:Package version is 19.0.0-811.geb1b049f 2024-01-21T22:15:00.119 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240119.075157/eb1b049f9003ac37e0a3ab33496ad1924d301254/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240119.075157/eb1b049f9003ac37e0a3ab33496ad1924d301254/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240119.075157/eb1b049f9003ac37e0a3ab33496ad1924d301254/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-21T22:15:00.119 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:15:00.119 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-21T22:15:00.206 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240119.075157/eb1b049f9003ac37e0a3ab33496ad1924d301254/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240119.075157/eb1b049f9003ac37e0a3ab33496ad1924d301254/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240119.075157/eb1b049f9003ac37e0a3ab33496ad1924d301254/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-01-21T22:15:00.207 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-21T22:15:00.207 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-01-21T22:15:00.214 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-01-21T22:15:00.215 DEBUG:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/eb1b049f9003ac37e0a3ab33496ad1924d301254/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-21T22:15:00.235 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-01-21T22:15:00.235 DEBUG:teuthology.orchestra.run.smithi152:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/eb1b049f9003ac37e0a3ab33496ad1924d301254/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-01-21T22:15:00.304 DEBUG:teuthology.orchestra.run.smithi152:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-21T22:15:00.352 DEBUG:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-01-21T22:15:00.385 DEBUG:teuthology.orchestra.run.smithi152:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-21T22:15:00.452 INFO:teuthology.orchestra.run.smithi152.stdout:check_obsoletes = 1 2024-01-21T22:15:00.453 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum clean all 2024-01-21T22:15:00.484 DEBUG:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-01-21T22:15:00.572 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2024-01-21T22:15:00.577 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum clean all 2024-01-21T22:15:00.688 INFO:teuthology.orchestra.run.smithi152.stdout:73 files removed 2024-01-21T22:15:00.715 DEBUG:teuthology.orchestra.run.smithi152:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd ceph-base-debuginfo ceph-common-debuginfo ceph-immutable-object-cache-debuginfo ceph-radosgw-debuginfo ceph-test-debuginfo ceph-base-debuginfo ceph-mgr-debuginfo ceph-mds-debuginfo ceph-mon-debuginfo ceph-osd-debuginfo ceph-fuse-debuginfo librados-devel-debuginfo libcephfs2-debuginfo librados2-debuginfo librbd1-debuginfo python3-cephfs-debuginfo python3-rados-debuginfo python3-rbd-debuginfo python3-rgw-debuginfo rbd-fuse-debuginfo rbd-mirror-debuginfo rbd-nbd-debuginfo python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-01-21T22:15:00.912 INFO:teuthology.orchestra.run.smithi138.stdout:73 files removed 2024-01-21T22:15:00.946 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd ceph-base-debuginfo ceph-common-debuginfo ceph-immutable-object-cache-debuginfo ceph-radosgw-debuginfo ceph-test-debuginfo ceph-base-debuginfo ceph-mgr-debuginfo ceph-mds-debuginfo ceph-mon-debuginfo ceph-osd-debuginfo ceph-fuse-debuginfo librados-devel-debuginfo libcephfs2-debuginfo librados2-debuginfo librbd1-debuginfo python3-cephfs-debuginfo python3-rados-debuginfo python3-rbd-debuginfo python3-rgw-debuginfo rbd-fuse-debuginfo rbd-mirror-debuginfo rbd-nbd-debuginfo python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-01-21T22:15:01.461 INFO:teuthology.orchestra.run.smithi152.stdout:ceph packages for x86_64 175 kB/s | 80 kB 00:00 2024-01-21T22:15:01.803 INFO:teuthology.orchestra.run.smithi152.stdout:ceph noarch packages 31 kB/s | 9.5 kB 00:00 2024-01-21T22:15:01.927 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 165 kB/s | 80 kB 00:00 2024-01-21T22:15:02.171 INFO:teuthology.orchestra.run.smithi152.stdout:ceph source packages 5.5 kB/s | 1.9 kB 00:00 2024-01-21T22:15:02.495 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 20 kB/s | 9.5 kB 00:00 2024-01-21T22:15:02.598 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS-9 - AppStream 16 MB/s | 6.4 MB 00:00 2024-01-21T22:15:02.937 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-01-21T22:15:03.524 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS-9 - AppStream 12 MB/s | 6.4 MB 00:00 2024-01-21T22:15:03.957 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS-9 - BaseOS 6.4 MB/s | 2.0 MB 00:00 2024-01-21T22:15:05.157 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS-9 - BaseOS 6.3 MB/s | 2.0 MB 00:00 2024-01-21T22:15:05.606 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - BaseOS 6.0 MB/s | 8.0 MB 00:01 2024-01-21T22:15:06.220 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-01-21T22:15:07.541 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-01-21T22:15:11.392 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - CRB 13 MB/s | 5.9 MB 00:00 2024-01-21T22:15:12.976 INFO:teuthology.orchestra.run.smithi152.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 15 kB 00:00 2024-01-21T22:15:15.744 INFO:teuthology.orchestra.run.smithi152.stdout:Extra Packages for Enterprise Linux 7.6 MB/s | 20 MB 00:02 2024-01-21T22:15:20.585 INFO:teuthology.orchestra.run.smithi152.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-01-21T22:15:23.082 INFO:teuthology.orchestra.run.smithi152.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-21T22:15:23.083 INFO:teuthology.orchestra.run.smithi152.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-21T22:15:23.116 INFO:teuthology.orchestra.run.smithi152.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-01-21T22:15:23.117 INFO:teuthology.orchestra.run.smithi152.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-01-21T22:15:23.118 INFO:teuthology.orchestra.run.smithi152.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-01-21T22:15:23.119 INFO:teuthology.orchestra.run.smithi152.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-01-21T22:15:23.121 INFO:teuthology.orchestra.run.smithi152.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-01-21T22:15:23.124 INFO:teuthology.orchestra.run.smithi152.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-01-21T22:15:23.125 INFO:teuthology.orchestra.run.smithi152.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-01-21T22:15:23.203 INFO:teuthology.orchestra.run.smithi152.stdout:Dependencies resolved. 2024-01-21T22:15:23.230 INFO:teuthology.orchestra.run.smithi152.stdout:=================================================================================================== 2024-01-21T22:15:23.230 INFO:teuthology.orchestra.run.smithi152.stdout: Package Arch Version Repository Size 2024-01-21T22:15:23.230 INFO:teuthology.orchestra.run.smithi152.stdout:=================================================================================================== 2024-01-21T22:15:23.230 INFO:teuthology.orchestra.run.smithi152.stdout:Installing: 2024-01-21T22:15:23.230 INFO:teuthology.orchestra.run.smithi152.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-01-21T22:15:23.230 INFO:teuthology.orchestra.run.smithi152.stdout: ceph x86_64 2:19.0.0-811.geb1b049f.el9 ceph 6.6 k 2024-01-21T22:15:23.230 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base x86_64 2:19.0.0-811.geb1b049f.el9 ceph 5.1 M 2024-01-21T22:15:23.230 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 67 M 2024-01-21T22:15:23.230 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 411 M 2024-01-21T22:15:23.230 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse x86_64 2:19.0.0-811.geb1b049f.el9 ceph 885 k 2024-01-21T22:15:23.230 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 5.9 M 2024-01-21T22:15:23.231 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-811.geb1b049f.el9 ceph 145 k 2024-01-21T22:15:23.231 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 1.4 M 2024-01-21T22:15:23.231 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 48 M 2024-01-21T22:15:23.231 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr x86_64 2:19.0.0-811.geb1b049f.el9 ceph 1.5 M 2024-01-21T22:15:23.231 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm noarch 2:19.0.0-811.geb1b049f.el9 ceph-noarch 133 k 2024-01-21T22:15:23.231 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard noarch 2:19.0.0-811.geb1b049f.el9 ceph-noarch 1.7 M 2024-01-21T22:15:23.231 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 20 M 2024-01-21T22:15:23.231 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-811.geb1b049f.el9 ceph-noarch 7.4 M 2024-01-21T22:15:23.231 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook noarch 2:19.0.0-811.geb1b049f.el9 ceph-noarch 51 k 2024-01-21T22:15:23.231 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 99 M 2024-01-21T22:15:23.231 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 286 M 2024-01-21T22:15:23.231 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw x86_64 2:19.0.0-811.geb1b049f.el9 ceph 8.6 M 2024-01-21T22:15:23.231 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 170 M 2024-01-21T22:15:23.232 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test x86_64 2:19.0.0-811.geb1b049f.el9 ceph 46 M 2024-01-21T22:15:23.232 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 664 M 2024-01-21T22:15:23.232 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-volume noarch 2:19.0.0-811.geb1b049f.el9 ceph-noarch 262 k 2024-01-21T22:15:23.232 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm noarch 2:19.0.0-811.geb1b049f.el9 ceph-noarch 742 k 2024-01-21T22:15:23.232 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-mirror x86_64 2:19.0.0-811.geb1b049f.el9 ceph 220 k 2024-01-21T22:15:23.232 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-shell x86_64 2:19.0.0-811.geb1b049f.el9 ceph 27 k 2024-01-21T22:15:23.232 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-top noarch 2:19.0.0-811.geb1b049f.el9 ceph-noarch 18 k 2024-01-21T22:15:23.232 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-01-21T22:15:23.232 INFO:teuthology.orchestra.run.smithi152.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-01-21T22:15:23.232 INFO:teuthology.orchestra.run.smithi152.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-01-21T22:15:23.232 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel x86_64 2:19.0.0-811.geb1b049f.el9 ceph 31 k 2024-01-21T22:15:23.232 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2 x86_64 2:19.0.0-811.geb1b049f.el9 ceph 743 k 2024-01-21T22:15:23.232 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 5.3 M 2024-01-21T22:15:23.232 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel x86_64 2:19.0.0-811.geb1b049f.el9 ceph 126 k 2024-01-21T22:15:23.233 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 108 k 2024-01-21T22:15:23.233 INFO:teuthology.orchestra.run.smithi152.stdout: librados2-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 44 M 2024-01-21T22:15:23.233 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 52 M 2024-01-21T22:15:23.233 INFO:teuthology.orchestra.run.smithi152.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-01-21T22:15:23.233 INFO:teuthology.orchestra.run.smithi152.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2024-01-21T22:15:23.233 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-01-21T22:15:23.233 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-01-21T22:15:23.233 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-01-21T22:15:23.233 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-01-21T22:15:23.233 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs x86_64 2:19.0.0-811.geb1b049f.el9 ceph 161 k 2024-01-21T22:15:23.233 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 615 k 2024-01-21T22:15:23.233 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados x86_64 2:19.0.0-811.geb1b049f.el9 ceph 321 k 2024-01-21T22:15:23.233 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 770 k 2024-01-21T22:15:23.233 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd x86_64 2:19.0.0-811.geb1b049f.el9 ceph 298 k 2024-01-21T22:15:23.233 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 1.1 M 2024-01-21T22:15:23.234 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw x86_64 2:19.0.0-811.geb1b049f.el9 ceph 99 k 2024-01-21T22:15:23.234 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 337 k 2024-01-21T22:15:23.234 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse x86_64 2:19.0.0-811.geb1b049f.el9 ceph 85 k 2024-01-21T22:15:23.234 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 509 k 2024-01-21T22:15:23.234 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror x86_64 2:19.0.0-811.geb1b049f.el9 ceph 3.1 M 2024-01-21T22:15:23.234 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 52 M 2024-01-21T22:15:23.234 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd x86_64 2:19.0.0-811.geb1b049f.el9 ceph 170 k 2024-01-21T22:15:23.234 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 1.3 M 2024-01-21T22:15:23.234 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-01-21T22:15:23.234 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrading: 2024-01-21T22:15:23.234 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager x86_64 1:1.45.10-1.el9 CentOS-BaseOS 2.2 M 2024-01-21T22:15:23.234 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-libnm x86_64 1:1.45.10-1.el9 CentOS-BaseOS 1.8 M 2024-01-21T22:15:23.234 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-team x86_64 1:1.45.10-1.el9 CentOS-BaseOS 42 k 2024-01-21T22:15:23.234 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-tui x86_64 1:1.45.10-1.el9 CentOS-BaseOS 246 k 2024-01-21T22:15:23.234 INFO:teuthology.orchestra.run.smithi152.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-01-21T22:15:23.235 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-01-21T22:15:23.235 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-01-21T22:15:23.235 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-01-21T22:15:23.235 INFO:teuthology.orchestra.run.smithi152.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-01-21T22:15:23.235 INFO:teuthology.orchestra.run.smithi152.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-01-21T22:15:23.235 INFO:teuthology.orchestra.run.smithi152.stdout: librados2 x86_64 2:19.0.0-811.geb1b049f.el9 ceph 3.4 M 2024-01-21T22:15:23.235 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1 x86_64 2:19.0.0-811.geb1b049f.el9 ceph 3.1 M 2024-01-21T22:15:23.235 INFO:teuthology.orchestra.run.smithi152.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-01-21T22:15:23.235 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-01-21T22:15:23.235 INFO:teuthology.orchestra.run.smithi152.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-01-21T22:15:23.235 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-01-21T22:15:23.235 INFO:teuthology.orchestra.run.smithi152.stdout:Installing dependencies: 2024-01-21T22:15:23.235 INFO:teuthology.orchestra.run.smithi152.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-01-21T22:15:23.235 INFO:teuthology.orchestra.run.smithi152.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-01-21T22:15:23.235 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common x86_64 2:19.0.0-811.geb1b049f.el9 ceph 19 M 2024-01-21T22:15:23.236 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 22 M 2024-01-21T22:15:23.236 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards noarch 2:19.0.0-811.geb1b049f.el9 ceph-noarch 28 k 2024-01-21T22:15:23.236 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds x86_64 2:19.0.0-811.geb1b049f.el9 ceph 2.1 M 2024-01-21T22:15:23.236 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core noarch 2:19.0.0-811.geb1b049f.el9 ceph-noarch 245 k 2024-01-21T22:15:23.236 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon x86_64 2:19.0.0-811.geb1b049f.el9 ceph 4.7 M 2024-01-21T22:15:23.236 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd x86_64 2:19.0.0-811.geb1b049f.el9 ceph 17 M 2024-01-21T22:15:23.236 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts noarch 2:19.0.0-811.geb1b049f.el9 ceph-noarch 15 k 2024-01-21T22:15:23.236 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux x86_64 2:19.0.0-811.geb1b049f.el9 ceph 25 k 2024-01-21T22:15:23.236 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-01-21T22:15:23.236 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-01-21T22:15:23.236 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-01-21T22:15:23.236 INFO:teuthology.orchestra.run.smithi152.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-01-21T22:15:23.236 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-01-21T22:15:23.236 INFO:teuthology.orchestra.run.smithi152.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-01-21T22:15:23.237 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-01-21T22:15:23.237 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-01-21T22:15:23.237 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite x86_64 2:19.0.0-811.geb1b049f.el9 ceph 163 k 2024-01-21T22:15:23.237 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-01-21T22:15:23.237 INFO:teuthology.orchestra.run.smithi152.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-01-21T22:15:23.237 INFO:teuthology.orchestra.run.smithi152.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-01-21T22:15:23.237 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-01-21T22:15:23.237 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-01-21T22:15:23.237 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1 x86_64 2:19.0.0-811.geb1b049f.el9 ceph 498 k 2024-01-21T22:15:23.237 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-01-21T22:15:23.237 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2 x86_64 2:19.0.0-811.geb1b049f.el9 ceph 4.8 M 2024-01-21T22:15:23.237 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-01-21T22:15:23.237 INFO:teuthology.orchestra.run.smithi152.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-01-21T22:15:23.237 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-01-21T22:15:23.237 INFO:teuthology.orchestra.run.smithi152.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-01-21T22:15:23.238 INFO:teuthology.orchestra.run.smithi152.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-01-21T22:15:23.238 INFO:teuthology.orchestra.run.smithi152.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-01-21T22:15:23.238 INFO:teuthology.orchestra.run.smithi152.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-01-21T22:15:23.238 INFO:teuthology.orchestra.run.smithi152.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-01-21T22:15:23.238 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-01-21T22:15:23.238 INFO:teuthology.orchestra.run.smithi152.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-01-21T22:15:23.238 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-01-21T22:15:23.238 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-01-21T22:15:23.238 INFO:teuthology.orchestra.run.smithi152.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-01-21T22:15:23.238 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-01-21T22:15:23.238 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-01-21T22:15:23.238 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-01-21T22:15:23.238 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse x86_64 2:19.0.0-811.geb1b049f.el9 ceph 45 k 2024-01-21T22:15:23.238 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common x86_64 2:19.0.0-811.geb1b049f.el9 ceph 130 k 2024-01-21T22:15:23.238 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-01-21T22:15:23.238 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-01-21T22:15:23.239 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-01-21T22:15:23.239 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-01-21T22:15:23.239 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-01-21T22:15:23.239 INFO:teuthology.orchestra.run.smithi152.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-01-21T22:15:23.239 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-01-21T22:15:23.239 INFO:teuthology.orchestra.run.smithi152.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-01-21T22:15:23.239 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth noarch 1:2.26.2-1.el9 epel 202 k 2024-01-21T22:15:23.239 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-01-21T22:15:23.239 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-01-21T22:15:23.239 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-01-21T22:15:23.239 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-01-21T22:15:23.239 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-01-21T22:15:23.239 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2024-01-21T22:15:23.239 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-01-21T22:15:23.239 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-01-21T22:15:23.239 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-01-21T22:15:23.240 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-01-21T22:15:23.240 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-01-21T22:15:23.240 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-01-21T22:15:23.240 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-01-21T22:15:23.240 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-01-21T22:15:23.240 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-01-21T22:15:23.240 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-01-21T22:15:23.240 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-01-21T22:15:23.240 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-01-21T22:15:23.240 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-01-21T22:15:23.240 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-01-21T22:15:23.240 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-01-21T22:15:23.240 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-01-21T22:15:23.240 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-01-21T22:15:23.241 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-01-21T22:15:23.241 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-01-21T22:15:23.241 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-01-21T22:15:23.241 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-01-21T22:15:23.241 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-01-21T22:15:23.241 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2024-01-21T22:15:23.241 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-01-21T22:15:23.241 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-01-21T22:15:23.241 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-01-21T22:15:23.241 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-01-21T22:15:23.241 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-01-21T22:15:23.241 INFO:teuthology.orchestra.run.smithi152.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-01-21T22:15:23.241 INFO:teuthology.orchestra.run.smithi152.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-01-21T22:15:23.241 INFO:teuthology.orchestra.run.smithi152.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-01-21T22:15:23.242 INFO:teuthology.orchestra.run.smithi152.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-01-21T22:15:23.242 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-01-21T22:15:23.242 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-01-21T22:15:23.242 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-01-21T22:15:23.242 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-01-21T22:15:23.242 INFO:teuthology.orchestra.run.smithi152.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-01-21T22:15:23.242 INFO:teuthology.orchestra.run.smithi152.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-01-21T22:15:23.242 INFO:teuthology.orchestra.run.smithi152.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-01-21T22:15:23.242 INFO:teuthology.orchestra.run.smithi152.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-01-21T22:15:23.242 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-01-21T22:15:23.242 INFO:teuthology.orchestra.run.smithi152.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-01-21T22:15:23.242 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:15:23.243 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction Summary 2024-01-21T22:15:23.243 INFO:teuthology.orchestra.run.smithi152.stdout:=================================================================================================== 2024-01-21T22:15:23.243 INFO:teuthology.orchestra.run.smithi152.stdout:Install 161 Packages 2024-01-21T22:15:23.243 INFO:teuthology.orchestra.run.smithi152.stdout:Upgrade 16 Packages 2024-01-21T22:15:23.243 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:15:23.243 INFO:teuthology.orchestra.run.smithi152.stdout:Total download size: 2.1 G 2024-01-21T22:15:23.243 INFO:teuthology.orchestra.run.smithi152.stdout:Downloading Packages: 2024-01-21T22:15:24.070 INFO:teuthology.orchestra.run.smithi152.stdout:(1/177): ceph-19.0.0-811.geb1b049f.el9.x86_64.r 37 kB/s | 6.6 kB 00:00 2024-01-21T22:15:24.578 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 19 MB 00:17 2024-01-21T22:15:25.079 INFO:teuthology.orchestra.run.smithi152.stdout:(2/177): ceph-common-19.0.0-811.geb1b049f.el9.x 19 MB/s | 19 MB 00:01 2024-01-21T22:15:26.512 INFO:teuthology.orchestra.run.smithi152.stdout:(3/177): ceph-base-19.0.0-811.geb1b049f.el9.x86 2.0 MB/s | 5.1 MB 00:02 2024-01-21T22:15:28.820 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 12 MB/s | 5.9 MB 00:00 2024-01-21T22:15:31.679 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 9.4 kB/s | 15 kB 00:01 2024-01-21T22:15:34.229 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 8.5 MB/s | 20 MB 00:02 2024-01-21T22:15:34.303 INFO:teuthology.orchestra.run.smithi152.stdout:(4/177): ceph-base-debuginfo-19.0.0-811.geb1b04 6.4 MB/s | 67 MB 00:10 2024-01-21T22:15:34.803 INFO:teuthology.orchestra.run.smithi152.stdout:(5/177): ceph-fuse-19.0.0-811.geb1b049f.el9.x86 1.7 MB/s | 885 kB 00:00 2024-01-21T22:15:35.670 INFO:teuthology.orchestra.run.smithi152.stdout:(6/177): ceph-debuginfo-19.0.0-811.geb1b049f.el 2.4 MB/s | 22 MB 00:09 2024-01-21T22:15:35.862 INFO:teuthology.orchestra.run.smithi152.stdout:(7/177): ceph-immutable-object-cache-19.0.0-811 759 kB/s | 145 kB 00:00 2024-01-21T22:15:36.271 INFO:teuthology.orchestra.run.smithi152.stdout:(8/177): ceph-fuse-debuginfo-19.0.0-811.geb1b04 4.0 MB/s | 5.9 MB 00:01 2024-01-21T22:15:36.788 INFO:teuthology.orchestra.run.smithi152.stdout:(9/177): ceph-immutable-object-cache-debuginfo- 1.5 MB/s | 1.4 MB 00:00 2024-01-21T22:15:36.838 INFO:teuthology.orchestra.run.smithi152.stdout:(10/177): ceph-mds-19.0.0-811.geb1b049f.el9.x86 3.7 MB/s | 2.1 MB 00:00 2024-01-21T22:15:37.214 INFO:teuthology.orchestra.run.smithi152.stdout:(11/177): ceph-mgr-19.0.0-811.geb1b049f.el9.x86 4.0 MB/s | 1.5 MB 00:00 2024-01-21T22:15:40.016 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-01-21T22:15:42.768 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-21T22:15:42.770 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-01-21T22:15:42.818 INFO:teuthology.orchestra.run.smithi138.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-01-21T22:15:42.819 INFO:teuthology.orchestra.run.smithi138.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-01-21T22:15:42.820 INFO:teuthology.orchestra.run.smithi138.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-01-21T22:15:42.821 INFO:teuthology.orchestra.run.smithi138.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-01-21T22:15:42.824 INFO:teuthology.orchestra.run.smithi138.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-01-21T22:15:42.827 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-01-21T22:15:42.828 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-01-21T22:15:42.912 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-01-21T22:15:42.928 INFO:teuthology.orchestra.run.smithi138.stdout:=================================================================================================== 2024-01-21T22:15:42.928 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-01-21T22:15:42.928 INFO:teuthology.orchestra.run.smithi138.stdout:=================================================================================================== 2024-01-21T22:15:42.929 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-01-21T22:15:42.929 INFO:teuthology.orchestra.run.smithi138.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-01-21T22:15:42.929 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_64 2:19.0.0-811.geb1b049f.el9 ceph 6.6 k 2024-01-21T22:15:42.929 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_64 2:19.0.0-811.geb1b049f.el9 ceph 5.1 M 2024-01-21T22:15:42.929 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 67 M 2024-01-21T22:15:42.929 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 411 M 2024-01-21T22:15:42.929 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_64 2:19.0.0-811.geb1b049f.el9 ceph 885 k 2024-01-21T22:15:42.929 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 5.9 M 2024-01-21T22:15:42.929 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-811.geb1b049f.el9 ceph 145 k 2024-01-21T22:15:42.929 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 1.4 M 2024-01-21T22:15:42.929 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 48 M 2024-01-21T22:15:42.929 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_64 2:19.0.0-811.geb1b049f.el9 ceph 1.5 M 2024-01-21T22:15:42.930 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm noarch 2:19.0.0-811.geb1b049f.el9 ceph-noarch 133 k 2024-01-21T22:15:42.930 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard noarch 2:19.0.0-811.geb1b049f.el9 ceph-noarch 1.7 M 2024-01-21T22:15:42.931 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 20 M 2024-01-21T22:15:42.931 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-811.geb1b049f.el9 ceph-noarch 7.4 M 2024-01-21T22:15:42.931 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook noarch 2:19.0.0-811.geb1b049f.el9 ceph-noarch 51 k 2024-01-21T22:15:42.931 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 99 M 2024-01-21T22:15:42.931 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 286 M 2024-01-21T22:15:42.931 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_64 2:19.0.0-811.geb1b049f.el9 ceph 8.6 M 2024-01-21T22:15:42.931 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 170 M 2024-01-21T22:15:42.931 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_64 2:19.0.0-811.geb1b049f.el9 ceph 46 M 2024-01-21T22:15:42.931 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 664 M 2024-01-21T22:15:42.931 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume noarch 2:19.0.0-811.geb1b049f.el9 ceph-noarch 262 k 2024-01-21T22:15:42.931 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm noarch 2:19.0.0-811.geb1b049f.el9 ceph-noarch 742 k 2024-01-21T22:15:42.932 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror x86_64 2:19.0.0-811.geb1b049f.el9 ceph 220 k 2024-01-21T22:15:42.932 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell x86_64 2:19.0.0-811.geb1b049f.el9 ceph 27 k 2024-01-21T22:15:42.932 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top noarch 2:19.0.0-811.geb1b049f.el9 ceph-noarch 18 k 2024-01-21T22:15:42.932 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 CentOS-AppStream 20 k 2024-01-21T22:15:42.932 INFO:teuthology.orchestra.run.smithi138.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-01-21T22:15:42.932 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-01-21T22:15:42.932 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_64 2:19.0.0-811.geb1b049f.el9 ceph 31 k 2024-01-21T22:15:42.932 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_64 2:19.0.0-811.geb1b049f.el9 ceph 743 k 2024-01-21T22:15:42.932 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 5.3 M 2024-01-21T22:15:42.932 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_64 2:19.0.0-811.geb1b049f.el9 ceph 126 k 2024-01-21T22:15:42.932 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 108 k 2024-01-21T22:15:42.933 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 44 M 2024-01-21T22:15:42.933 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 52 M 2024-01-21T22:15:42.933 INFO:teuthology.orchestra.run.smithi138.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-01-21T22:15:42.933 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2024-01-21T22:15:42.933 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel x86_64 1:3.0.7-25.el9 CentOS-AppStream 3.0 M 2024-01-21T22:15:42.933 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-01-21T22:15:42.933 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-01-21T22:15:42.933 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-01-21T22:15:42.933 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_64 2:19.0.0-811.geb1b049f.el9 ceph 161 k 2024-01-21T22:15:42.933 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 615 k 2024-01-21T22:15:42.933 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_64 2:19.0.0-811.geb1b049f.el9 ceph 321 k 2024-01-21T22:15:42.933 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 770 k 2024-01-21T22:15:42.934 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_64 2:19.0.0-811.geb1b049f.el9 ceph 298 k 2024-01-21T22:15:42.934 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 1.1 M 2024-01-21T22:15:42.934 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_64 2:19.0.0-811.geb1b049f.el9 ceph 99 k 2024-01-21T22:15:42.934 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 337 k 2024-01-21T22:15:42.934 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_64 2:19.0.0-811.geb1b049f.el9 ceph 85 k 2024-01-21T22:15:42.934 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 509 k 2024-01-21T22:15:42.934 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_64 2:19.0.0-811.geb1b049f.el9 ceph 3.1 M 2024-01-21T22:15:42.934 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 52 M 2024-01-21T22:15:42.934 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_64 2:19.0.0-811.geb1b049f.el9 ceph 170 k 2024-01-21T22:15:42.934 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 1.3 M 2024-01-21T22:15:42.934 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 CentOS-AppStream 87 k 2024-01-21T22:15:42.935 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2024-01-21T22:15:42.935 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager x86_64 1:1.45.10-1.el9 CentOS-BaseOS 2.2 M 2024-01-21T22:15:42.935 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-libnm x86_64 1:1.45.10-1.el9 CentOS-BaseOS 1.8 M 2024-01-21T22:15:42.935 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-team x86_64 1:1.45.10-1.el9 CentOS-BaseOS 42 k 2024-01-21T22:15:42.935 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-tui x86_64 1:1.45.10-1.el9 CentOS-BaseOS 246 k 2024-01-21T22:15:42.935 INFO:teuthology.orchestra.run.smithi138.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2024-01-21T22:15:42.935 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 CentOS-BaseOS 35 k 2024-01-21T22:15:42.935 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf x86_64 0.190-2.el9 CentOS-BaseOS 191 k 2024-01-21T22:15:42.935 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libs x86_64 0.190-2.el9 CentOS-BaseOS 254 k 2024-01-21T22:15:42.935 INFO:teuthology.orchestra.run.smithi138.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2024-01-21T22:15:42.935 INFO:teuthology.orchestra.run.smithi138.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2024-01-21T22:15:42.935 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_64 2:19.0.0-811.geb1b049f.el9 ceph 3.4 M 2024-01-21T22:15:42.936 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_64 2:19.0.0-811.geb1b049f.el9 ceph 3.1 M 2024-01-21T22:15:42.936 INFO:teuthology.orchestra.run.smithi138.stdout: openssl x86_64 1:3.0.7-25.el9 CentOS-BaseOS 1.2 M 2024-01-21T22:15:42.936 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs x86_64 1:3.0.7-25.el9 CentOS-BaseOS 2.1 M 2024-01-21T22:15:42.936 INFO:teuthology.orchestra.run.smithi138.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-01-21T22:15:42.936 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs x86_64 6.3.0-1.el9 CentOS-BaseOS 1.1 M 2024-01-21T22:15:42.936 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-01-21T22:15:42.936 INFO:teuthology.orchestra.run.smithi138.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-01-21T22:15:42.936 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-01-21T22:15:42.936 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_64 2:19.0.0-811.geb1b049f.el9 ceph 19 M 2024-01-21T22:15:42.936 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-debuginfo x86_64 2:19.0.0-811.geb1b049f.el9 ceph 22 M 2024-01-21T22:15:42.936 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards noarch 2:19.0.0-811.geb1b049f.el9 ceph-noarch 28 k 2024-01-21T22:15:42.937 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_64 2:19.0.0-811.geb1b049f.el9 ceph 2.1 M 2024-01-21T22:15:42.937 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch 2:19.0.0-811.geb1b049f.el9 ceph-noarch 245 k 2024-01-21T22:15:42.937 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_64 2:19.0.0-811.geb1b049f.el9 ceph 4.7 M 2024-01-21T22:15:42.937 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_64 2:19.0.0-811.geb1b049f.el9 ceph 17 M 2024-01-21T22:15:42.937 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts noarch 2:19.0.0-811.geb1b049f.el9 ceph-noarch 15 k 2024-01-21T22:15:42.937 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_64 2:19.0.0-811.geb1b049f.el9 ceph 25 k 2024-01-21T22:15:42.937 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-01-21T22:15:42.937 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-01-21T22:15:42.937 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-01-21T22:15:42.937 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-01-21T22:15:42.937 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-01-21T22:15:42.938 INFO:teuthology.orchestra.run.smithi138.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-01-21T22:15:42.938 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-01-21T22:15:42.938 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-01-21T22:15:42.938 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite x86_64 2:19.0.0-811.geb1b049f.el9 ceph 163 k 2024-01-21T22:15:42.938 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-01-21T22:15:42.938 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-01-21T22:15:42.938 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-01-21T22:15:42.938 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-01-21T22:15:42.938 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-01-21T22:15:42.938 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_64 2:19.0.0-811.geb1b049f.el9 ceph 498 k 2024-01-21T22:15:42.938 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-01-21T22:15:42.938 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_64 2:19.0.0-811.geb1b049f.el9 ceph 4.8 M 2024-01-21T22:15:42.939 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-01-21T22:15:42.939 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-01-21T22:15:42.939 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-01-21T22:15:42.939 INFO:teuthology.orchestra.run.smithi138.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-01-21T22:15:42.939 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-01-21T22:15:42.939 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-01-21T22:15:42.939 INFO:teuthology.orchestra.run.smithi138.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-01-21T22:15:42.939 INFO:teuthology.orchestra.run.smithi138.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-01-21T22:15:42.939 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-01-21T22:15:42.939 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-01-21T22:15:42.939 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-01-21T22:15:42.939 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-01-21T22:15:42.940 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-01-21T22:15:42.940 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-01-21T22:15:42.940 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-01-21T22:15:42.940 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-01-21T22:15:42.940 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse x86_64 2:19.0.0-811.geb1b049f.el9 ceph 45 k 2024-01-21T22:15:42.940 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_64 2:19.0.0-811.geb1b049f.el9 ceph 130 k 2024-01-21T22:15:42.940 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-01-21T22:15:42.940 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-01-21T22:15:42.940 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-01-21T22:15:42.940 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-01-21T22:15:42.940 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-01-21T22:15:42.941 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-01-21T22:15:42.941 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-01-21T22:15:42.941 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-01-21T22:15:42.941 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth noarch 1:2.26.2-1.el9 epel 202 k 2024-01-21T22:15:42.941 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-01-21T22:15:42.941 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-01-21T22:15:42.941 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-01-21T22:15:42.941 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-01-21T22:15:42.941 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-01-21T22:15:42.941 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2024-01-21T22:15:42.941 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-01-21T22:15:42.941 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-01-21T22:15:42.942 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-01-21T22:15:42.942 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-01-21T22:15:42.942 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-01-21T22:15:42.942 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-01-21T22:15:42.942 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-01-21T22:15:42.942 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-01-21T22:15:42.942 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-01-21T22:15:42.942 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-01-21T22:15:42.942 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-01-21T22:15:42.942 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-01-21T22:15:42.942 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-01-21T22:15:42.942 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-01-21T22:15:42.943 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-01-21T22:15:42.943 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-01-21T22:15:42.943 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-01-21T22:15:42.943 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-01-21T22:15:42.943 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-01-21T22:15:42.943 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-01-21T22:15:42.943 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-01-21T22:15:42.943 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-01-21T22:15:42.943 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2024-01-21T22:15:42.943 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-01-21T22:15:42.943 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-01-21T22:15:42.943 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-01-21T22:15:42.944 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-01-21T22:15:42.944 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-01-21T22:15:42.944 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-01-21T22:15:42.944 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-01-21T22:15:42.944 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-01-21T22:15:42.944 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-01-21T22:15:42.944 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-01-21T22:15:42.944 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-01-21T22:15:42.944 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-01-21T22:15:42.945 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-01-21T22:15:42.945 INFO:teuthology.orchestra.run.smithi138.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-01-21T22:15:42.945 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-01-21T22:15:42.945 INFO:teuthology.orchestra.run.smithi138.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-01-21T22:15:42.945 INFO:teuthology.orchestra.run.smithi138.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-01-21T22:15:42.946 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-01-21T22:15:42.946 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-01-21T22:15:42.946 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:15:42.946 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-01-21T22:15:42.946 INFO:teuthology.orchestra.run.smithi138.stdout:=================================================================================================== 2024-01-21T22:15:42.946 INFO:teuthology.orchestra.run.smithi138.stdout:Install 161 Packages 2024-01-21T22:15:42.946 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 16 Packages 2024-01-21T22:15:42.946 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:15:42.948 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 2.1 G 2024-01-21T22:15:42.948 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-01-21T22:15:44.243 INFO:teuthology.orchestra.run.smithi138.stdout:(1/177): ceph-19.0.0-811.geb1b049f.el9.x86_64.r 16 kB/s | 6.6 kB 00:00 2024-01-21T22:15:46.642 INFO:teuthology.orchestra.run.smithi138.stdout:(2/177): ceph-base-19.0.0-811.geb1b049f.el9.x86 1.8 MB/s | 5.1 MB 00:02 2024-01-21T22:15:47.846 INFO:teuthology.orchestra.run.smithi152.stdout:(12/177): ceph-mgr-debuginfo-19.0.0-811.geb1b04 1.8 MB/s | 20 MB 00:10 2024-01-21T22:15:50.621 INFO:teuthology.orchestra.run.smithi152.stdout:(13/177): ceph-mon-19.0.0-811.geb1b049f.el9.x86 1.7 MB/s | 4.7 MB 00:02 2024-01-21T22:15:54.275 INFO:teuthology.orchestra.run.smithi138.stdout:(3/177): ceph-common-19.0.0-811.geb1b049f.el9.x 1.9 MB/s | 19 MB 00:10 2024-01-21T22:15:54.971 INFO:teuthology.orchestra.run.smithi152.stdout:(14/177): ceph-mds-debuginfo-19.0.0-811.geb1b04 2.6 MB/s | 48 MB 00:18 2024-01-21T22:15:58.071 INFO:teuthology.orchestra.run.smithi152.stdout:(15/177): ceph-osd-19.0.0-811.geb1b049f.el9.x86 5.4 MB/s | 17 MB 00:03 2024-01-21T22:16:02.441 INFO:teuthology.orchestra.run.smithi138.stdout:(4/177): ceph-debuginfo-19.0.0-811.geb1b049f.el 2.7 MB/s | 22 MB 00:08 2024-01-21T22:16:02.825 INFO:teuthology.orchestra.run.smithi138.stdout:(5/177): ceph-fuse-19.0.0-811.geb1b049f.el9.x86 2.3 MB/s | 885 kB 00:00 2024-01-21T22:16:04.808 INFO:teuthology.orchestra.run.smithi138.stdout:(6/177): ceph-fuse-debuginfo-19.0.0-811.geb1b04 3.0 MB/s | 5.9 MB 00:01 2024-01-21T22:16:04.900 INFO:teuthology.orchestra.run.smithi138.stdout:(7/177): ceph-immutable-object-cache-19.0.0-811 1.6 MB/s | 145 kB 00:00 2024-01-21T22:16:05.425 INFO:teuthology.orchestra.run.smithi138.stdout:(8/177): ceph-immutable-object-cache-debuginfo- 2.7 MB/s | 1.4 MB 00:00 2024-01-21T22:16:06.184 INFO:teuthology.orchestra.run.smithi138.stdout:(9/177): ceph-mds-19.0.0-811.geb1b049f.el9.x86_ 2.8 MB/s | 2.1 MB 00:00 2024-01-21T22:16:09.434 INFO:teuthology.orchestra.run.smithi138.stdout:(10/177): ceph-base-debuginfo-19.0.0-811.geb1b0 2.6 MB/s | 67 MB 00:25 2024-01-21T22:16:10.259 INFO:teuthology.orchestra.run.smithi138.stdout:(11/177): ceph-mgr-19.0.0-811.geb1b049f.el9.x86 1.8 MB/s | 1.5 MB 00:00 2024-01-21T22:16:13.228 INFO:teuthology.orchestra.run.smithi152.stdout:(16/177): ceph-common-debuginfo-19.0.0-811.geb1 8.5 MB/s | 411 MB 00:48 2024-01-21T22:16:13.239 INFO:teuthology.orchestra.run.smithi138.stdout:(12/177): ceph-mds-debuginfo-19.0.0-811.geb1b04 6.8 MB/s | 48 MB 00:07 2024-01-21T22:16:13.810 INFO:teuthology.orchestra.run.smithi138.stdout:(13/177): ceph-mon-19.0.0-811.geb1b049f.el9.x86 8.2 MB/s | 4.7 MB 00:00 2024-01-21T22:16:14.295 INFO:teuthology.orchestra.run.smithi152.stdout:(17/177): ceph-radosgw-19.0.0-811.geb1b049f.el9 8.1 MB/s | 8.6 MB 00:01 2024-01-21T22:16:18.568 INFO:teuthology.orchestra.run.smithi138.stdout:(14/177): ceph-mgr-debuginfo-19.0.0-811.geb1b04 2.4 MB/s | 20 MB 00:08 2024-01-21T22:16:22.759 INFO:teuthology.orchestra.run.smithi138.stdout:(15/177): ceph-osd-19.0.0-811.geb1b049f.el9.x86 4.0 MB/s | 17 MB 00:04 2024-01-21T22:16:28.142 INFO:teuthology.orchestra.run.smithi138.stdout:(16/177): ceph-mon-debuginfo-19.0.0-811.geb1b04 6.9 MB/s | 99 MB 00:14 2024-01-21T22:16:30.130 INFO:teuthology.orchestra.run.smithi138.stdout:(17/177): ceph-radosgw-19.0.0-811.geb1b049f.el9 4.3 MB/s | 8.6 MB 00:01 2024-01-21T22:16:30.676 INFO:teuthology.orchestra.run.smithi152.stdout:(18/177): ceph-mon-debuginfo-19.0.0-811.geb1b04 2.5 MB/s | 99 MB 00:40 2024-01-21T22:16:30.768 INFO:teuthology.orchestra.run.smithi152.stdout:(19/177): ceph-selinux-19.0.0-811.geb1b049f.el9 274 kB/s | 25 kB 00:00 2024-01-21T22:16:39.509 INFO:teuthology.orchestra.run.smithi152.stdout:(20/177): ceph-test-19.0.0-811.geb1b049f.el9.x8 5.2 MB/s | 46 MB 00:08 2024-01-21T22:16:51.558 INFO:teuthology.orchestra.run.smithi152.stdout:(21/177): ceph-osd-debuginfo-19.0.0-811.geb1b04 5.4 MB/s | 286 MB 00:53 2024-01-21T22:16:52.108 INFO:teuthology.orchestra.run.smithi152.stdout:(22/177): cephfs-mirror-19.0.0-811.geb1b049f.el 401 kB/s | 220 kB 00:00 2024-01-21T22:16:52.258 INFO:teuthology.orchestra.run.smithi152.stdout:(23/177): cephfs-shell-19.0.0-811.geb1b049f.el9 180 kB/s | 27 kB 00:00 2024-01-21T22:16:52.384 INFO:teuthology.orchestra.run.smithi152.stdout:(24/177): libcephfs-devel-19.0.0-811.geb1b049f. 249 kB/s | 31 kB 00:00 2024-01-21T22:16:52.759 INFO:teuthology.orchestra.run.smithi152.stdout:(25/177): libcephfs2-19.0.0-811.geb1b049f.el9.x 1.9 MB/s | 743 kB 00:00 2024-01-21T22:16:55.175 INFO:teuthology.orchestra.run.smithi152.stdout:(26/177): libcephfs2-debuginfo-19.0.0-811.geb1b 2.2 MB/s | 5.3 MB 00:02 2024-01-21T22:16:55.292 INFO:teuthology.orchestra.run.smithi152.stdout:(27/177): libcephsqlite-19.0.0-811.geb1b049f.el 1.4 MB/s | 163 kB 00:00 2024-01-21T22:16:55.401 INFO:teuthology.orchestra.run.smithi152.stdout:(28/177): librados-devel-19.0.0-811.geb1b049f.e 1.1 MB/s | 126 kB 00:00 2024-01-21T22:16:55.501 INFO:teuthology.orchestra.run.smithi152.stdout:(29/177): librados-devel-debuginfo-19.0.0-811.g 1.1 MB/s | 108 kB 00:00 2024-01-21T22:16:59.485 INFO:teuthology.orchestra.run.smithi152.stdout:(30/177): ceph-radosgw-debuginfo-19.0.0-811.geb 3.8 MB/s | 170 MB 00:45 2024-01-21T22:16:59.860 INFO:teuthology.orchestra.run.smithi152.stdout:(31/177): libradosstriper1-19.0.0-811.geb1b049f 1.3 MB/s | 498 kB 00:00 2024-01-21T22:17:11.734 INFO:teuthology.orchestra.run.smithi152.stdout:(32/177): librados2-debuginfo-19.0.0-811.geb1b0 2.7 MB/s | 44 MB 00:16 2024-01-21T22:17:13.225 INFO:teuthology.orchestra.run.smithi152.stdout:(33/177): librgw2-19.0.0-811.geb1b049f.el9.x86_ 3.2 MB/s | 4.8 MB 00:01 2024-01-21T22:17:13.359 INFO:teuthology.orchestra.run.smithi152.stdout:(34/177): python3-ceph-argparse-19.0.0-811.geb1 339 kB/s | 45 kB 00:00 2024-01-21T22:17:13.459 INFO:teuthology.orchestra.run.smithi152.stdout:(35/177): python3-ceph-common-19.0.0-811.geb1b0 1.3 MB/s | 130 kB 00:00 2024-01-21T22:17:13.543 INFO:teuthology.orchestra.run.smithi152.stdout:(36/177): python3-cephfs-19.0.0-811.geb1b049f.e 1.9 MB/s | 161 kB 00:00 2024-01-21T22:17:13.578 INFO:teuthology.orchestra.run.smithi138.stdout:(18/177): ceph-radosgw-debuginfo-19.0.0-811.geb 3.9 MB/s | 170 MB 00:43 2024-01-21T22:17:13.662 INFO:teuthology.orchestra.run.smithi138.stdout:(19/177): ceph-selinux-19.0.0-811.geb1b049f.el9 306 kB/s | 25 kB 00:00 2024-01-21T22:17:13.801 INFO:teuthology.orchestra.run.smithi152.stdout:(37/177): python3-cephfs-debuginfo-19.0.0-811.g 2.3 MB/s | 615 kB 00:00 2024-01-21T22:17:13.985 INFO:teuthology.orchestra.run.smithi152.stdout:(38/177): python3-rados-19.0.0-811.geb1b049f.el 1.7 MB/s | 321 kB 00:00 2024-01-21T22:17:14.294 INFO:teuthology.orchestra.run.smithi152.stdout:(39/177): python3-rados-debuginfo-19.0.0-811.ge 2.4 MB/s | 770 kB 00:00 2024-01-21T22:17:14.411 INFO:teuthology.orchestra.run.smithi152.stdout:(40/177): python3-rbd-19.0.0-811.geb1b049f.el9. 2.5 MB/s | 298 kB 00:00 2024-01-21T22:17:14.802 INFO:teuthology.orchestra.run.smithi152.stdout:(41/177): python3-rbd-debuginfo-19.0.0-811.geb1 2.9 MB/s | 1.1 MB 00:00 2024-01-21T22:17:14.903 INFO:teuthology.orchestra.run.smithi152.stdout:(42/177): python3-rgw-19.0.0-811.geb1b049f.el9. 993 kB/s | 99 kB 00:00 2024-01-21T22:17:15.103 INFO:teuthology.orchestra.run.smithi152.stdout:(43/177): python3-rgw-debuginfo-19.0.0-811.geb1 1.6 MB/s | 337 kB 00:00 2024-01-21T22:17:15.212 INFO:teuthology.orchestra.run.smithi152.stdout:(44/177): rbd-fuse-19.0.0-811.geb1b049f.el9.x86 786 kB/s | 85 kB 00:00 2024-01-21T22:17:15.795 INFO:teuthology.orchestra.run.smithi152.stdout:(45/177): librbd1-debuginfo-19.0.0-811.geb1b049 3.3 MB/s | 52 MB 00:15 2024-01-21T22:17:15.820 INFO:teuthology.orchestra.run.smithi152.stdout:(46/177): rbd-fuse-debuginfo-19.0.0-811.geb1b04 836 kB/s | 509 kB 00:00 2024-01-21T22:17:16.846 INFO:teuthology.orchestra.run.smithi152.stdout:(47/177): rbd-mirror-19.0.0-811.geb1b049f.el9.x 2.9 MB/s | 3.1 MB 00:01 2024-01-21T22:17:16.921 INFO:teuthology.orchestra.run.smithi152.stdout:(48/177): rbd-nbd-19.0.0-811.geb1b049f.el9.x86_ 2.2 MB/s | 170 kB 00:00 2024-01-21T22:17:17.247 INFO:teuthology.orchestra.run.smithi152.stdout:(49/177): rbd-nbd-debuginfo-19.0.0-811.geb1b049 3.8 MB/s | 1.3 MB 00:00 2024-01-21T22:17:17.314 INFO:teuthology.orchestra.run.smithi152.stdout:(50/177): ceph-grafana-dashboards-19.0.0-811.ge 426 kB/s | 28 kB 00:00 2024-01-21T22:17:17.381 INFO:teuthology.orchestra.run.smithi152.stdout:(51/177): ceph-mgr-cephadm-19.0.0-811.geb1b049f 2.0 MB/s | 133 kB 00:00 2024-01-21T22:17:17.806 INFO:teuthology.orchestra.run.smithi152.stdout:(52/177): ceph-mgr-dashboard-19.0.0-811.geb1b04 4.1 MB/s | 1.7 MB 00:00 2024-01-21T22:17:19.773 INFO:teuthology.orchestra.run.smithi152.stdout:(53/177): ceph-mgr-diskprediction-local-19.0.0- 3.8 MB/s | 7.4 MB 00:01 2024-01-21T22:17:19.906 INFO:teuthology.orchestra.run.smithi152.stdout:(54/177): ceph-mgr-modules-core-19.0.0-811.geb1 1.8 MB/s | 245 kB 00:00 2024-01-21T22:17:20.006 INFO:teuthology.orchestra.run.smithi152.stdout:(55/177): ceph-mgr-rook-19.0.0-811.geb1b049f.el 508 kB/s | 51 kB 00:00 2024-01-21T22:17:20.082 INFO:teuthology.orchestra.run.smithi152.stdout:(56/177): ceph-prometheus-alerts-19.0.0-811.geb 201 kB/s | 15 kB 00:00 2024-01-21T22:17:20.215 INFO:teuthology.orchestra.run.smithi152.stdout:(57/177): ceph-volume-19.0.0-811.geb1b049f.el9. 1.9 MB/s | 262 kB 00:00 2024-01-21T22:17:20.432 INFO:teuthology.orchestra.run.smithi152.stdout:(58/177): cephadm-19.0.0-811.geb1b049f.el9.noar 3.3 MB/s | 742 kB 00:00 2024-01-21T22:17:20.508 INFO:teuthology.orchestra.run.smithi152.stdout:(59/177): cephfs-top-19.0.0-811.geb1b049f.el9.n 244 kB/s | 18 kB 00:00 2024-01-21T22:17:20.691 INFO:teuthology.orchestra.run.smithi152.stdout:(60/177): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 662 kB 00:00 2024-01-21T22:17:20.758 INFO:teuthology.orchestra.run.smithi152.stdout:(61/177): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 920 kB 00:00 2024-01-21T22:17:20.800 INFO:teuthology.orchestra.run.smithi152.stdout:(62/177): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 106 kB 00:00 2024-01-21T22:17:20.834 INFO:teuthology.orchestra.run.smithi152.stdout:(63/177): elfutils-libelf-devel-0.190-2.el9.x86 616 kB/s | 20 kB 00:00 2024-01-21T22:17:20.876 INFO:teuthology.orchestra.run.smithi152.stdout:(64/177): flex-2.6.4-9.el9.x86_64.rpm 7.2 MB/s | 307 kB 00:00 2024-01-21T22:17:20.909 INFO:teuthology.orchestra.run.smithi152.stdout:(65/177): flexiblas-3.0.4-8.el9.x86_64.rpm 953 kB/s | 32 kB 00:00 2024-01-21T22:17:21.043 INFO:teuthology.orchestra.run.smithi152.stdout:(66/177): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-01-21T22:17:21.077 INFO:teuthology.orchestra.run.smithi152.stdout:(67/177): flexiblas-openblas-openmp-3.0.4-8.el9 514 kB/s | 17 kB 00:00 2024-01-21T22:17:21.119 INFO:teuthology.orchestra.run.smithi152.stdout:(68/177): libaio-devel-0.3.111-13.el9.x86_64.rp 241 kB/s | 10 kB 00:00 2024-01-21T22:17:21.161 INFO:teuthology.orchestra.run.smithi152.stdout:(69/177): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 159 kB 00:00 2024-01-21T22:17:21.195 INFO:teuthology.orchestra.run.smithi152.stdout:(70/177): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-01-21T22:17:21.245 INFO:teuthology.orchestra.run.smithi152.stdout:(71/177): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-01-21T22:17:21.295 INFO:teuthology.orchestra.run.smithi152.stdout:(72/177): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2024-01-21T22:17:21.329 INFO:teuthology.orchestra.run.smithi152.stdout:(73/177): libuuid-devel-2.37.4-15.el9.x86_64.rp 729 kB/s | 24 kB 00:00 2024-01-21T22:17:21.363 INFO:teuthology.orchestra.run.smithi152.stdout:(74/177): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-01-21T22:17:21.405 INFO:teuthology.orchestra.run.smithi152.stdout:(75/177): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.6 MB/s | 282 kB 00:00 2024-01-21T22:17:21.447 INFO:teuthology.orchestra.run.smithi152.stdout:(76/177): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2024-01-21T22:17:21.489 INFO:teuthology.orchestra.run.smithi152.stdout:(77/177): openblas-0.3.21-2.el9.x86_64.rpm 820 kB/s | 34 kB 00:00 2024-01-21T22:17:21.656 INFO:teuthology.orchestra.run.smithi152.stdout:(78/177): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-01-21T22:17:21.773 INFO:teuthology.orchestra.run.smithi152.stdout:(79/177): openssl-devel-3.0.7-25.el9.x86_64.rpm 26 MB/s | 3.0 MB 00:00 2024-01-21T22:17:21.856 INFO:teuthology.orchestra.run.smithi152.stdout:(80/177): postgresql-13.11-1.el9.x86_64.rpm 18 MB/s | 1.5 MB 00:00 2024-01-21T22:17:21.923 INFO:teuthology.orchestra.run.smithi152.stdout:(81/177): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2024-01-21T22:17:21.965 INFO:teuthology.orchestra.run.smithi152.stdout:(82/177): postgresql-private-libs-13.11-1.el9.x 3.1 MB/s | 134 kB 00:00 2024-01-21T22:17:22.166 INFO:teuthology.orchestra.run.smithi152.stdout:(83/177): postgresql-server-13.11-1.el9.x86_64. 28 MB/s | 5.7 MB 00:00 2024-01-21T22:17:22.208 INFO:teuthology.orchestra.run.smithi152.stdout:(84/177): python3-attrs-20.3.0-7.el9.noarch.rpm 1.8 MB/s | 78 kB 00:00 2024-01-21T22:17:22.433 INFO:teuthology.orchestra.run.smithi152.stdout:(85/177): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 5.8 MB 00:00 2024-01-21T22:17:22.475 INFO:teuthology.orchestra.run.smithi152.stdout:(86/177): python3-jinja2-2.11.3-4.el9.noarch.rp 5.3 MB/s | 228 kB 00:00 2024-01-21T22:17:22.509 INFO:teuthology.orchestra.run.smithi152.stdout:(87/177): python3-jsonpatch-1.21-16.el9.noarch. 734 kB/s | 24 kB 00:00 2024-01-21T22:17:22.542 INFO:teuthology.orchestra.run.smithi152.stdout:(88/177): python3-jsonpointer-2.0-4.el9.noarch. 545 kB/s | 18 kB 00:00 2024-01-21T22:17:22.584 INFO:teuthology.orchestra.run.smithi152.stdout:(89/177): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-01-21T22:17:22.626 INFO:teuthology.orchestra.run.smithi152.stdout:(90/177): python3-markupsafe-1.1.1-12.el9.x86_6 760 kB/s | 32 kB 00:00 2024-01-21T22:17:22.818 INFO:teuthology.orchestra.run.smithi152.stdout:(91/177): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 4.8 MB 00:00 2024-01-21T22:17:22.868 INFO:teuthology.orchestra.run.smithi152.stdout:(92/177): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 217 kB 00:00 2024-01-21T22:17:22.927 INFO:teuthology.orchestra.run.smithi152.stdout:(93/177): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 174 kB 00:00 2024-01-21T22:17:22.969 INFO:teuthology.orchestra.run.smithi152.stdout:(94/177): python3-prettytable-0.7.2-27.el9.noar 984 kB/s | 41 kB 00:00 2024-01-21T22:17:22.978 INFO:teuthology.orchestra.run.smithi138.stdout:(20/177): ceph-test-19.0.0-811.geb1b049f.el9.x8 4.9 MB/s | 46 MB 00:09 2024-01-21T22:17:23.019 INFO:teuthology.orchestra.run.smithi152.stdout:(95/177): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-01-21T22:17:23.070 INFO:teuthology.orchestra.run.smithi152.stdout:(96/177): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-01-21T22:17:23.128 INFO:teuthology.orchestra.run.smithi152.stdout:(97/177): python3-pytz-2021.1-5.el9.noarch.rpm 811 kB/s | 47 kB 00:00 2024-01-21T22:17:23.179 INFO:teuthology.orchestra.run.smithi152.stdout:(98/177): python3-requests-oauthlib-1.3.0-12.el 846 kB/s | 43 kB 00:00 2024-01-21T22:17:23.621 INFO:teuthology.orchestra.run.smithi152.stdout:(99/177): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-01-21T22:17:23.654 INFO:teuthology.orchestra.run.smithi152.stdout:(100/177): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-01-21T22:17:23.688 INFO:teuthology.orchestra.run.smithi152.stdout:(101/177): python3-wcwidth-0.2.5-8.el9.noarch.r 1.2 MB/s | 41 kB 00:00 2024-01-21T22:17:23.730 INFO:teuthology.orchestra.run.smithi152.stdout:(102/177): socat-1.7.4.1-5.el9.x86_64.rpm 7.0 MB/s | 300 kB 00:00 2024-01-21T22:17:23.764 INFO:teuthology.orchestra.run.smithi152.stdout:(103/177): uuid-1.6.2-55.el9.x86_64.rpm 1.6 MB/s | 56 kB 00:00 2024-01-21T22:17:23.797 INFO:teuthology.orchestra.run.smithi152.stdout:(104/177): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 2.5 MB/s | 87 kB 00:00 2024-01-21T22:17:23.831 INFO:teuthology.orchestra.run.smithi152.stdout:(105/177): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-01-21T22:17:23.865 INFO:teuthology.orchestra.run.smithi152.stdout:(106/177): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-01-21T22:17:23.898 INFO:teuthology.orchestra.run.smithi152.stdout:(107/177): fuse-2.9.9-15.el9.x86_64.rpm 2.3 MB/s | 79 kB 00:00 2024-01-21T22:17:23.940 INFO:teuthology.orchestra.run.smithi152.stdout:(108/177): jq-1.6-16.el9.x86_64.rpm 4.4 MB/s | 186 kB 00:00 2024-01-21T22:17:23.999 INFO:teuthology.orchestra.run.smithi152.stdout:(109/177): libgfortran-11.4.1-3.el9.x86_64.rpm 13 MB/s | 802 kB 00:00 2024-01-21T22:17:24.050 INFO:teuthology.orchestra.run.smithi152.stdout:(110/177): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 194 kB 00:00 2024-01-21T22:17:24.091 INFO:teuthology.orchestra.run.smithi152.stdout:(111/177): oniguruma-6.9.6-1.el9.5.x86_64.rpm 5.1 MB/s | 217 kB 00:00 2024-01-21T22:17:24.133 INFO:teuthology.orchestra.run.smithi152.stdout:(112/177): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.6 MB/s | 241 kB 00:00 2024-01-21T22:17:24.192 INFO:teuthology.orchestra.run.smithi152.stdout:(113/177): python3-cryptography-36.0.1-4.el9.x8 20 MB/s | 1.1 MB 00:00 2024-01-21T22:17:24.234 INFO:teuthology.orchestra.run.smithi152.stdout:(114/177): python3-ply-3.11-14.el9.noarch.rpm 2.4 MB/s | 103 kB 00:00 2024-01-21T22:17:24.267 INFO:teuthology.orchestra.run.smithi152.stdout:(115/177): python3-pycparser-2.20-6.el9.noarch. 3.6 MB/s | 124 kB 00:00 2024-01-21T22:17:24.326 INFO:teuthology.orchestra.run.smithi152.stdout:(116/177): python3-requests-2.25.1-7.el9.noarch 1.9 MB/s | 113 kB 00:00 2024-01-21T22:17:24.377 INFO:teuthology.orchestra.run.smithi152.stdout:(117/177): python3-urllib3-1.26.5-5.el9.noarch. 3.7 MB/s | 187 kB 00:00 2024-01-21T22:17:24.635 INFO:teuthology.orchestra.run.smithi152.stdout:(118/177): python3-devel-3.9.17-2.el9.x86_64.rp 950 kB/s | 245 kB 00:00 2024-01-21T22:17:25.277 INFO:teuthology.orchestra.run.smithi152.stdout:(119/177): lua-devel-5.4.4-4.el9.x86_64.rpm 35 kB/s | 22 kB 00:00 2024-01-21T22:17:25.469 INFO:teuthology.orchestra.run.smithi152.stdout:(120/177): gperftools-libs-2.9.1-2.el9.x86_64.r 1.6 MB/s | 309 kB 00:00 2024-01-21T22:17:25.728 INFO:teuthology.orchestra.run.smithi152.stdout:(121/177): libarrow-9.0.0-10.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-01-21T22:17:25.778 INFO:teuthology.orchestra.run.smithi152.stdout:(122/177): libarrow-doc-9.0.0-10.el9.noarch.rpm 520 kB/s | 26 kB 00:00 2024-01-21T22:17:25.828 INFO:teuthology.orchestra.run.smithi152.stdout:(123/177): liboath-2.6.7-2.el9.x86_64.rpm 970 kB/s | 49 kB 00:00 2024-01-21T22:17:25.879 INFO:teuthology.orchestra.run.smithi152.stdout:(124/177): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-01-21T22:17:25.929 INFO:teuthology.orchestra.run.smithi152.stdout:(125/177): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-01-21T22:17:26.021 INFO:teuthology.orchestra.run.smithi152.stdout:(126/177): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 838 kB 00:00 2024-01-21T22:17:26.088 INFO:teuthology.orchestra.run.smithi152.stdout:(127/177): python3-asyncssh-2.13.2-3.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-01-21T22:17:26.139 INFO:teuthology.orchestra.run.smithi152.stdout:(128/177): python3-bcrypt-3.2.2-1.el9.x86_64.rp 867 kB/s | 43 kB 00:00 2024-01-21T22:17:26.152 INFO:teuthology.orchestra.run.smithi138.stdout:(21/177): ceph-osd-debuginfo-19.0.0-811.geb1b04 4.5 MB/s | 286 MB 01:03 2024-01-21T22:17:26.189 INFO:teuthology.orchestra.run.smithi152.stdout:(129/177): python3-cachetools-4.2.4-1.el9.noarc 644 kB/s | 32 kB 00:00 2024-01-21T22:17:26.239 INFO:teuthology.orchestra.run.smithi152.stdout:(130/177): python3-certifi-2021.10.8-2.el9.noar 301 kB/s | 15 kB 00:00 2024-01-21T22:17:26.298 INFO:teuthology.orchestra.run.smithi152.stdout:(131/177): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-01-21T22:17:26.356 INFO:teuthology.orchestra.run.smithi152.stdout:(132/177): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-01-21T22:17:26.415 INFO:teuthology.orchestra.run.smithi152.stdout:(133/177): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-01-21T22:17:26.465 INFO:teuthology.orchestra.run.smithi152.stdout:(134/177): python3-colorama-0.4.4-12.el9.noarch 704 kB/s | 35 kB 00:00 2024-01-21T22:17:26.469 INFO:teuthology.orchestra.run.smithi138.stdout:(22/177): cephfs-mirror-19.0.0-811.geb1b049f.el 699 kB/s | 220 kB 00:00 2024-01-21T22:17:26.516 INFO:teuthology.orchestra.run.smithi152.stdout:(135/177): python3-google-auth-2.26.2-1.el9.noa 3.9 MB/s | 202 kB 00:00 2024-01-21T22:17:26.562 INFO:teuthology.orchestra.run.smithi138.stdout:(23/177): cephfs-shell-19.0.0-811.geb1b049f.el9 299 kB/s | 27 kB 00:00 2024-01-21T22:17:26.566 INFO:teuthology.orchestra.run.smithi152.stdout:(136/177): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-01-21T22:17:26.616 INFO:teuthology.orchestra.run.smithi152.stdout:(137/177): python3-jaraco-classes-3.2.1-5.el9.n 354 kB/s | 18 kB 00:00 2024-01-21T22:17:26.645 INFO:teuthology.orchestra.run.smithi138.stdout:(24/177): libcephfs-devel-19.0.0-811.geb1b049f. 378 kB/s | 31 kB 00:00 2024-01-21T22:17:26.667 INFO:teuthology.orchestra.run.smithi152.stdout:(138/177): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-01-21T22:17:26.717 INFO:teuthology.orchestra.run.smithi152.stdout:(139/177): python3-jaraco-functools-3.5.0-2.el9 389 kB/s | 19 kB 00:00 2024-01-21T22:17:26.767 INFO:teuthology.orchestra.run.smithi152.stdout:(140/177): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-01-21T22:17:26.851 INFO:teuthology.orchestra.run.smithi152.stdout:(141/177): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-01-21T22:17:26.902 INFO:teuthology.orchestra.run.smithi152.stdout:(142/177): python3-logutils-0.3.5-21.el9.noarch 928 kB/s | 46 kB 00:00 2024-01-21T22:17:26.927 INFO:teuthology.orchestra.run.smithi138.stdout:(25/177): libcephfs2-19.0.0-811.geb1b049f.el9.x 2.6 MB/s | 743 kB 00:00 2024-01-21T22:17:26.952 INFO:teuthology.orchestra.run.smithi152.stdout:(143/177): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-01-21T22:17:27.003 INFO:teuthology.orchestra.run.smithi152.stdout:(144/177): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-01-21T22:17:27.061 INFO:teuthology.orchestra.run.smithi152.stdout:(145/177): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-01-21T22:17:27.111 INFO:teuthology.orchestra.run.smithi152.stdout:(146/177): python3-portend-3.1.0-2.el9.noarch.r 329 kB/s | 16 kB 00:00 2024-01-21T22:17:27.161 INFO:teuthology.orchestra.run.smithi152.stdout:(147/177): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-01-21T22:17:27.212 INFO:teuthology.orchestra.run.smithi152.stdout:(148/177): python3-pyperclip-1.8.2-2.el9.noarch 565 kB/s | 28 kB 00:00 2024-01-21T22:17:27.262 INFO:teuthology.orchestra.run.smithi152.stdout:(149/177): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-01-21T22:17:27.321 INFO:teuthology.orchestra.run.smithi152.stdout:(150/177): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-01-21T22:17:27.371 INFO:teuthology.orchestra.run.smithi152.stdout:(151/177): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-01-21T22:17:27.421 INFO:teuthology.orchestra.run.smithi152.stdout:(152/177): python3-tempora-5.0.0-2.el9.noarch.r 716 kB/s | 36 kB 00:00 2024-01-21T22:17:27.472 INFO:teuthology.orchestra.run.smithi152.stdout:(153/177): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-01-21T22:17:27.539 INFO:teuthology.orchestra.run.smithi152.stdout:(154/177): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-01-21T22:17:27.589 INFO:teuthology.orchestra.run.smithi152.stdout:(155/177): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-01-21T22:17:27.648 INFO:teuthology.orchestra.run.smithi152.stdout:(156/177): python3-werkzeug-2.0.3-3.el9.1.noarc 7.2 MB/s | 427 kB 00:00 2024-01-21T22:17:27.690 INFO:teuthology.orchestra.run.smithi152.stdout:(157/177): python3-zc-lockfile-2.0-10.el9.noarc 482 kB/s | 20 kB 00:00 2024-01-21T22:17:27.749 INFO:teuthology.orchestra.run.smithi152.stdout:(158/177): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-01-21T22:17:28.004 INFO:teuthology.orchestra.run.smithi138.stdout:(26/177): libcephfs2-debuginfo-19.0.0-811.geb1b 5.0 MB/s | 5.3 MB 00:01 2024-01-21T22:17:28.049 INFO:teuthology.orchestra.run.smithi152.stdout:(159/177): thrift-0.15.0-2.el9.x86_64.rpm 5.3 MB/s | 1.6 MB 00:00 2024-01-21T22:17:28.128 INFO:teuthology.orchestra.run.smithi138.stdout:(27/177): libcephsqlite-19.0.0-811.geb1b049f.el 1.3 MB/s | 163 kB 00:00 2024-01-21T22:17:28.238 INFO:teuthology.orchestra.run.smithi138.stdout:(28/177): librados-devel-19.0.0-811.geb1b049f.e 1.1 MB/s | 126 kB 00:00 2024-01-21T22:17:28.346 INFO:teuthology.orchestra.run.smithi138.stdout:(29/177): librados-devel-debuginfo-19.0.0-811.g 1.0 MB/s | 108 kB 00:00 2024-01-21T22:17:29.424 INFO:teuthology.orchestra.run.smithi152.stdout:(160/177): librados2-19.0.0-811.geb1b049f.el9.x 2.4 MB/s | 3.4 MB 00:01 2024-01-21T22:17:30.824 INFO:teuthology.orchestra.run.smithi152.stdout:(161/177): librbd1-19.0.0-811.geb1b049f.el9.x86 2.2 MB/s | 3.1 MB 00:01 2024-01-21T22:17:30.858 INFO:teuthology.orchestra.run.smithi152.stdout:(162/177): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2024-01-21T22:17:30.975 INFO:teuthology.orchestra.run.smithi152.stdout:(163/177): NetworkManager-1.45.10-1.el9.x86_64. 19 MB/s | 2.2 MB 00:00 2024-01-21T22:17:31.058 INFO:teuthology.orchestra.run.smithi152.stdout:(164/177): NetworkManager-libnm-1.45.10-1.el9.x 22 MB/s | 1.8 MB 00:00 2024-01-21T22:17:31.109 INFO:teuthology.orchestra.run.smithi152.stdout:(165/177): NetworkManager-team-1.45.10-1.el9.x8 844 kB/s | 42 kB 00:00 2024-01-21T22:17:31.167 INFO:teuthology.orchestra.run.smithi152.stdout:(166/177): NetworkManager-tui-1.45.10-1.el9.x86 4.1 MB/s | 246 kB 00:00 2024-01-21T22:17:31.209 INFO:teuthology.orchestra.run.smithi152.stdout:(167/177): acl-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 69 kB 00:00 2024-01-21T22:17:31.251 INFO:teuthology.orchestra.run.smithi152.stdout:(168/177): elfutils-debuginfod-client-0.190-2.e 853 kB/s | 35 kB 00:00 2024-01-21T22:17:31.293 INFO:teuthology.orchestra.run.smithi152.stdout:(169/177): elfutils-libelf-0.190-2.el9.x86_64.r 4.5 MB/s | 191 kB 00:00 2024-01-21T22:17:31.335 INFO:teuthology.orchestra.run.smithi152.stdout:(170/177): elfutils-libs-0.190-2.el9.x86_64.rpm 6.0 MB/s | 254 kB 00:00 2024-01-21T22:17:31.377 INFO:teuthology.orchestra.run.smithi152.stdout:(171/177): libacl-2.3.1-4.el9.x86_64.rpm 527 kB/s | 22 kB 00:00 2024-01-21T22:17:31.452 INFO:teuthology.orchestra.run.smithi152.stdout:(172/177): openssl-3.0.7-25.el9.x86_64.rpm 15 MB/s | 1.2 MB 00:00 2024-01-21T22:17:31.553 INFO:teuthology.orchestra.run.smithi152.stdout:(173/177): openssl-libs-3.0.7-25.el9.x86_64.rpm 21 MB/s | 2.1 MB 00:00 2024-01-21T22:17:31.603 INFO:teuthology.orchestra.run.smithi152.stdout:(174/177): xfsdump-3.1.12-4.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-01-21T22:17:31.662 INFO:teuthology.orchestra.run.smithi152.stdout:(175/177): xfsprogs-6.3.0-1.el9.x86_64.rpm 18 MB/s | 1.1 MB 00:00 2024-01-21T22:17:32.354 INFO:teuthology.orchestra.run.smithi152.stdout:(176/177): rbd-mirror-debuginfo-19.0.0-811.geb1 3.1 MB/s | 52 MB 00:16 2024-01-21T22:17:38.829 INFO:teuthology.orchestra.run.smithi138.stdout:(30/177): librados2-debuginfo-19.0.0-811.geb1b0 4.2 MB/s | 44 MB 00:10 2024-01-21T22:17:39.079 INFO:teuthology.orchestra.run.smithi138.stdout:(31/177): libradosstriper1-19.0.0-811.geb1b049f 2.0 MB/s | 498 kB 00:00 2024-01-21T22:17:46.654 INFO:teuthology.orchestra.run.smithi138.stdout:(32/177): ceph-common-debuginfo-19.0.0-811.geb1 3.4 MB/s | 411 MB 02:00 2024-01-21T22:17:47.912 INFO:teuthology.orchestra.run.smithi138.stdout:(33/177): librgw2-19.0.0-811.geb1b049f.el9.x86_ 3.8 MB/s | 4.8 MB 00:01 2024-01-21T22:17:48.004 INFO:teuthology.orchestra.run.smithi138.stdout:(34/177): python3-ceph-argparse-19.0.0-811.geb1 496 kB/s | 45 kB 00:00 2024-01-21T22:17:48.112 INFO:teuthology.orchestra.run.smithi138.stdout:(35/177): python3-ceph-common-19.0.0-811.geb1b0 1.2 MB/s | 130 kB 00:00 2024-01-21T22:17:48.238 INFO:teuthology.orchestra.run.smithi138.stdout:(36/177): python3-cephfs-19.0.0-811.geb1b049f.e 1.3 MB/s | 161 kB 00:00 2024-01-21T22:17:48.413 INFO:teuthology.orchestra.run.smithi138.stdout:(37/177): python3-cephfs-debuginfo-19.0.0-811.g 3.4 MB/s | 615 kB 00:00 2024-01-21T22:17:48.522 INFO:teuthology.orchestra.run.smithi138.stdout:(38/177): python3-rados-19.0.0-811.geb1b049f.el 2.9 MB/s | 321 kB 00:00 2024-01-21T22:17:48.722 INFO:teuthology.orchestra.run.smithi138.stdout:(39/177): python3-rados-debuginfo-19.0.0-811.ge 3.8 MB/s | 770 kB 00:00 2024-01-21T22:17:48.831 INFO:teuthology.orchestra.run.smithi138.stdout:(40/177): python3-rbd-19.0.0-811.geb1b049f.el9. 2.7 MB/s | 298 kB 00:00 2024-01-21T22:17:49.098 INFO:teuthology.orchestra.run.smithi138.stdout:(41/177): python3-rbd-debuginfo-19.0.0-811.geb1 4.2 MB/s | 1.1 MB 00:00 2024-01-21T22:17:49.190 INFO:teuthology.orchestra.run.smithi138.stdout:(42/177): python3-rgw-19.0.0-811.geb1b049f.el9. 1.1 MB/s | 99 kB 00:00 2024-01-21T22:17:49.306 INFO:teuthology.orchestra.run.smithi138.stdout:(43/177): python3-rgw-debuginfo-19.0.0-811.geb1 2.9 MB/s | 337 kB 00:00 2024-01-21T22:17:49.398 INFO:teuthology.orchestra.run.smithi138.stdout:(44/177): rbd-fuse-19.0.0-811.geb1b049f.el9.x86 939 kB/s | 85 kB 00:00 2024-01-21T22:17:49.574 INFO:teuthology.orchestra.run.smithi138.stdout:(45/177): rbd-fuse-debuginfo-19.0.0-811.geb1b04 2.9 MB/s | 509 kB 00:00 2024-01-21T22:17:50.191 INFO:teuthology.orchestra.run.smithi138.stdout:(46/177): rbd-mirror-19.0.0-811.geb1b049f.el9.x 5.0 MB/s | 3.1 MB 00:00 2024-01-21T22:17:50.850 INFO:teuthology.orchestra.run.smithi138.stdout:(47/177): librbd1-debuginfo-19.0.0-811.geb1b049 4.4 MB/s | 52 MB 00:11 2024-01-21T22:17:51.000 INFO:teuthology.orchestra.run.smithi138.stdout:(48/177): rbd-nbd-19.0.0-811.geb1b049f.el9.x86_ 1.1 MB/s | 170 kB 00:00 2024-01-21T22:17:51.342 INFO:teuthology.orchestra.run.smithi138.stdout:(49/177): rbd-nbd-debuginfo-19.0.0-811.geb1b049 3.7 MB/s | 1.3 MB 00:00 2024-01-21T22:17:51.442 INFO:teuthology.orchestra.run.smithi138.stdout:(50/177): ceph-grafana-dashboards-19.0.0-811.ge 287 kB/s | 28 kB 00:00 2024-01-21T22:17:51.534 INFO:teuthology.orchestra.run.smithi138.stdout:(51/177): ceph-mgr-cephadm-19.0.0-811.geb1b049f 1.4 MB/s | 133 kB 00:00 2024-01-21T22:17:51.918 INFO:teuthology.orchestra.run.smithi138.stdout:(52/177): ceph-mgr-dashboard-19.0.0-811.geb1b04 4.5 MB/s | 1.7 MB 00:00 2024-01-21T22:17:53.293 INFO:teuthology.orchestra.run.smithi138.stdout:(53/177): ceph-mgr-diskprediction-local-19.0.0- 5.4 MB/s | 7.4 MB 00:01 2024-01-21T22:17:53.410 INFO:teuthology.orchestra.run.smithi138.stdout:(54/177): ceph-mgr-modules-core-19.0.0-811.geb1 2.1 MB/s | 245 kB 00:00 2024-01-21T22:17:55.052 INFO:teuthology.orchestra.run.smithi138.stdout:(55/177): ceph-mgr-rook-19.0.0-811.geb1b049f.el 31 kB/s | 51 kB 00:01 2024-01-21T22:17:55.143 INFO:teuthology.orchestra.run.smithi138.stdout:(56/177): ceph-prometheus-alerts-19.0.0-811.geb 167 kB/s | 15 kB 00:00 2024-01-21T22:17:55.444 INFO:teuthology.orchestra.run.smithi138.stdout:(57/177): ceph-volume-19.0.0-811.geb1b049f.el9. 876 kB/s | 262 kB 00:00 2024-01-21T22:17:55.694 INFO:teuthology.orchestra.run.smithi138.stdout:(58/177): cephadm-19.0.0-811.geb1b049f.el9.noar 2.9 MB/s | 742 kB 00:00 2024-01-21T22:17:55.794 INFO:teuthology.orchestra.run.smithi138.stdout:(59/177): cephfs-top-19.0.0-811.geb1b049f.el9.n 184 kB/s | 18 kB 00:00 2024-01-21T22:17:55.978 INFO:teuthology.orchestra.run.smithi138.stdout:(60/177): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 662 kB 00:00 2024-01-21T22:17:56.045 INFO:teuthology.orchestra.run.smithi138.stdout:(61/177): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 920 kB 00:00 2024-01-21T22:17:56.087 INFO:teuthology.orchestra.run.smithi138.stdout:(62/177): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 106 kB 00:00 2024-01-21T22:17:56.112 INFO:teuthology.orchestra.run.smithi138.stdout:(63/177): elfutils-libelf-devel-0.190-2.el9.x86 851 kB/s | 20 kB 00:00 2024-01-21T22:17:56.162 INFO:teuthology.orchestra.run.smithi138.stdout:(64/177): flex-2.6.4-9.el9.x86_64.rpm 6.2 MB/s | 307 kB 00:00 2024-01-21T22:17:56.188 INFO:teuthology.orchestra.run.smithi138.stdout:(65/177): flexiblas-3.0.4-8.el9.x86_64.rpm 1.3 MB/s | 32 kB 00:00 2024-01-21T22:17:56.305 INFO:teuthology.orchestra.run.smithi138.stdout:(66/177): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-01-21T22:17:56.330 INFO:teuthology.orchestra.run.smithi138.stdout:(67/177): flexiblas-openblas-openmp-3.0.4-8.el9 722 kB/s | 17 kB 00:00 2024-01-21T22:17:56.355 INFO:teuthology.orchestra.run.smithi138.stdout:(68/177): libaio-devel-0.3.111-13.el9.x86_64.rp 419 kB/s | 10 kB 00:00 2024-01-21T22:17:56.397 INFO:teuthology.orchestra.run.smithi138.stdout:(69/177): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 159 kB 00:00 2024-01-21T22:17:56.431 INFO:teuthology.orchestra.run.smithi138.stdout:(70/177): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-01-21T22:17:56.481 INFO:teuthology.orchestra.run.smithi138.stdout:(71/177): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-01-21T22:17:56.540 INFO:teuthology.orchestra.run.smithi138.stdout:(72/177): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 578 kB 00:00 2024-01-21T22:17:56.574 INFO:teuthology.orchestra.run.smithi138.stdout:(73/177): libuuid-devel-2.37.4-15.el9.x86_64.rp 761 kB/s | 24 kB 00:00 2024-01-21T22:17:56.640 INFO:teuthology.orchestra.run.smithi138.stdout:(74/177): libzstd-devel-1.5.1-2.el9.x86_64.rpm 708 kB/s | 46 kB 00:00 2024-01-21T22:17:56.683 INFO:teuthology.orchestra.run.smithi138.stdout:(75/177): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.7 MB/s | 282 kB 00:00 2024-01-21T22:17:56.733 INFO:teuthology.orchestra.run.smithi138.stdout:(76/177): lua-5.4.4-4.el9.x86_64.rpm 3.8 MB/s | 187 kB 00:00 2024-01-21T22:17:56.775 INFO:teuthology.orchestra.run.smithi138.stdout:(77/177): openblas-0.3.21-2.el9.x86_64.rpm 843 kB/s | 34 kB 00:00 2024-01-21T22:17:56.934 INFO:teuthology.orchestra.run.smithi138.stdout:(78/177): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2024-01-21T22:17:57.076 INFO:teuthology.orchestra.run.smithi138.stdout:(79/177): openssl-devel-3.0.7-25.el9.x86_64.rpm 21 MB/s | 3.0 MB 00:00 2024-01-21T22:17:57.159 INFO:teuthology.orchestra.run.smithi138.stdout:(80/177): postgresql-13.11-1.el9.x86_64.rpm 18 MB/s | 1.5 MB 00:00 2024-01-21T22:17:57.226 INFO:teuthology.orchestra.run.smithi138.stdout:(81/177): postgresql-contrib-13.11-1.el9.x86_64 12 MB/s | 811 kB 00:00 2024-01-21T22:17:57.268 INFO:teuthology.orchestra.run.smithi138.stdout:(82/177): postgresql-private-libs-13.11-1.el9.x 3.2 MB/s | 134 kB 00:00 2024-01-21T22:17:57.452 INFO:teuthology.orchestra.run.smithi138.stdout:(83/177): postgresql-server-13.11-1.el9.x86_64. 31 MB/s | 5.7 MB 00:00 2024-01-21T22:17:57.494 INFO:teuthology.orchestra.run.smithi138.stdout:(84/177): python3-attrs-20.3.0-7.el9.noarch.rpm 1.9 MB/s | 78 kB 00:00 2024-01-21T22:17:57.661 INFO:teuthology.orchestra.run.smithi138.stdout:(85/177): python3-babel-2.9.1-2.el9.noarch.rpm 35 MB/s | 5.8 MB 00:00 2024-01-21T22:17:57.703 INFO:teuthology.orchestra.run.smithi138.stdout:(86/177): python3-jinja2-2.11.3-4.el9.noarch.rp 5.5 MB/s | 228 kB 00:00 2024-01-21T22:17:57.728 INFO:teuthology.orchestra.run.smithi138.stdout:(87/177): python3-jsonpatch-1.21-16.el9.noarch. 1.0 MB/s | 24 kB 00:00 2024-01-21T22:17:57.753 INFO:teuthology.orchestra.run.smithi138.stdout:(88/177): python3-jsonpointer-2.0-4.el9.noarch. 752 kB/s | 18 kB 00:00 2024-01-21T22:17:57.795 INFO:teuthology.orchestra.run.smithi138.stdout:(89/177): python3-mako-1.1.4-6.el9.noarch.rpm 3.5 MB/s | 144 kB 00:00 2024-01-21T22:17:57.854 INFO:teuthology.orchestra.run.smithi138.stdout:(90/177): python3-markupsafe-1.1.1-12.el9.x86_6 545 kB/s | 32 kB 00:00 2024-01-21T22:17:57.996 INFO:teuthology.orchestra.run.smithi138.stdout:(91/177): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2024-01-21T22:17:58.038 INFO:teuthology.orchestra.run.smithi138.stdout:(92/177): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.2 MB/s | 217 kB 00:00 2024-01-21T22:17:58.080 INFO:teuthology.orchestra.run.smithi138.stdout:(93/177): python3-oauthlib-3.1.1-5.el9.noarch.r 4.2 MB/s | 174 kB 00:00 2024-01-21T22:17:58.114 INFO:teuthology.orchestra.run.smithi138.stdout:(94/177): python3-prettytable-0.7.2-27.el9.noar 1.3 MB/s | 41 kB 00:00 2024-01-21T22:17:58.164 INFO:teuthology.orchestra.run.smithi138.stdout:(95/177): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-01-21T22:17:58.237 INFO:teuthology.orchestra.run.smithi138.stdout:(96/177): python3-pyasn1-modules-0.4.8-6.el9.no 2.9 MB/s | 211 kB 00:00 2024-01-21T22:17:58.273 INFO:teuthology.orchestra.run.smithi138.stdout:(97/177): python3-pytz-2021.1-5.el9.noarch.rpm 1.3 MB/s | 47 kB 00:00 2024-01-21T22:17:58.307 INFO:teuthology.orchestra.run.smithi138.stdout:(98/177): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 43 kB 00:00 2024-01-21T22:17:58.824 INFO:teuthology.orchestra.run.smithi138.stdout:(99/177): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-01-21T22:17:58.857 INFO:teuthology.orchestra.run.smithi138.stdout:(100/177): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 39 kB 00:00 2024-01-21T22:17:58.899 INFO:teuthology.orchestra.run.smithi138.stdout:(101/177): python3-wcwidth-0.2.5-8.el9.noarch.r 1.0 MB/s | 41 kB 00:00 2024-01-21T22:17:58.958 INFO:teuthology.orchestra.run.smithi138.stdout:(102/177): socat-1.7.4.1-5.el9.x86_64.rpm 5.1 MB/s | 300 kB 00:00 2024-01-21T22:17:59.575 INFO:teuthology.orchestra.run.smithi138.stdout:(103/177): rbd-mirror-debuginfo-19.0.0-811.geb1 5.5 MB/s | 52 MB 00:09 2024-01-21T22:17:59.600 INFO:teuthology.orchestra.run.smithi138.stdout:(104/177): uuid-1.6.2-55.el9.x86_64.rpm 88 kB/s | 56 kB 00:00 2024-01-21T22:17:59.625 INFO:teuthology.orchestra.run.smithi138.stdout:(105/177): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 87 kB 00:00 2024-01-21T22:17:59.650 INFO:teuthology.orchestra.run.smithi138.stdout:(106/177): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 63 kB 00:00 2024-01-21T22:17:59.684 INFO:teuthology.orchestra.run.smithi138.stdout:(107/177): zlib-devel-1.2.11-40.el9.x86_64.rpm 767 kB/s | 44 kB 00:00 2024-01-21T22:17:59.710 INFO:teuthology.orchestra.run.smithi138.stdout:(108/177): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 79 kB 00:00 2024-01-21T22:17:59.752 INFO:teuthology.orchestra.run.smithi138.stdout:(109/177): jq-1.6-16.el9.x86_64.rpm 2.8 MB/s | 186 kB 00:00 2024-01-21T22:17:59.802 INFO:teuthology.orchestra.run.smithi138.stdout:(110/177): libgfortran-11.4.1-3.el9.x86_64.rpm 8.6 MB/s | 802 kB 00:00 2024-01-21T22:17:59.836 INFO:teuthology.orchestra.run.smithi138.stdout:(111/177): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 194 kB 00:00 2024-01-21T22:17:59.869 INFO:teuthology.orchestra.run.smithi138.stdout:(112/177): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2024-01-21T22:17:59.903 INFO:teuthology.orchestra.run.smithi138.stdout:(113/177): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 241 kB 00:00 2024-01-21T22:17:59.978 INFO:teuthology.orchestra.run.smithi138.stdout:(114/177): python3-cryptography-36.0.1-4.el9.x8 11 MB/s | 1.1 MB 00:00 2024-01-21T22:18:00.004 INFO:teuthology.orchestra.run.smithi138.stdout:(115/177): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2024-01-21T22:18:00.029 INFO:teuthology.orchestra.run.smithi138.stdout:(116/177): python3-pycparser-2.20-6.el9.noarch. 2.5 MB/s | 124 kB 00:00 2024-01-21T22:18:00.054 INFO:teuthology.orchestra.run.smithi138.stdout:(117/177): python3-requests-2.25.1-7.el9.noarch 2.3 MB/s | 113 kB 00:00 2024-01-21T22:18:00.088 INFO:teuthology.orchestra.run.smithi138.stdout:(118/177): python3-urllib3-1.26.5-5.el9.noarch. 3.2 MB/s | 187 kB 00:00 2024-01-21T22:18:00.246 INFO:teuthology.orchestra.run.smithi138.stdout:(119/177): python3-devel-3.9.17-2.el9.x86_64.rp 1.3 MB/s | 245 kB 00:00 2024-01-21T22:18:00.305 INFO:teuthology.orchestra.run.smithi138.stdout:(120/177): lua-devel-5.4.4-4.el9.x86_64.rpm 103 kB/s | 22 kB 00:00 2024-01-21T22:18:00.522 INFO:teuthology.orchestra.run.smithi138.stdout:(121/177): gperftools-libs-2.9.1-2.el9.x86_64.r 1.1 MB/s | 309 kB 00:00 2024-01-21T22:18:00.581 INFO:teuthology.orchestra.run.smithi138.stdout:(122/177): libarrow-doc-9.0.0-10.el9.noarch.rpm 453 kB/s | 26 kB 00:00 2024-01-21T22:18:00.639 INFO:teuthology.orchestra.run.smithi138.stdout:(123/177): liboath-2.6.7-2.el9.x86_64.rpm 844 kB/s | 49 kB 00:00 2024-01-21T22:18:00.706 INFO:teuthology.orchestra.run.smithi138.stdout:(124/177): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-01-21T22:18:00.832 INFO:teuthology.orchestra.run.smithi138.stdout:(125/177): libarrow-9.0.0-10.el9.x86_64.rpm 8.4 MB/s | 4.4 MB 00:00 2024-01-21T22:18:00.857 INFO:teuthology.orchestra.run.smithi138.stdout:(126/177): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-01-21T22:18:00.924 INFO:teuthology.orchestra.run.smithi138.stdout:(127/177): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.0 MB/s | 838 kB 00:00 2024-01-21T22:18:00.982 INFO:teuthology.orchestra.run.smithi138.stdout:(128/177): python3-asyncssh-2.13.2-3.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-01-21T22:18:01.016 INFO:teuthology.orchestra.run.smithi138.stdout:(129/177): python3-bcrypt-3.2.2-1.el9.x86_64.rp 477 kB/s | 43 kB 00:00 2024-01-21T22:18:01.041 INFO:teuthology.orchestra.run.smithi138.stdout:(130/177): python3-cachetools-4.2.4-1.el9.noarc 559 kB/s | 32 kB 00:00 2024-01-21T22:18:01.075 INFO:teuthology.orchestra.run.smithi138.stdout:(131/177): python3-certifi-2021.10.8-2.el9.noar 259 kB/s | 15 kB 00:00 2024-01-21T22:18:01.108 INFO:teuthology.orchestra.run.smithi138.stdout:(132/177): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-01-21T22:18:01.167 INFO:teuthology.orchestra.run.smithi138.stdout:(133/177): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-01-21T22:18:01.201 INFO:teuthology.orchestra.run.smithi138.stdout:(134/177): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-01-21T22:18:01.235 INFO:teuthology.orchestra.run.smithi138.stdout:(135/177): python3-colorama-0.4.4-12.el9.noarch 532 kB/s | 35 kB 00:00 2024-01-21T22:18:01.268 INFO:teuthology.orchestra.run.smithi138.stdout:(136/177): python3-google-auth-2.26.2-1.el9.noa 3.0 MB/s | 202 kB 00:00 2024-01-21T22:18:01.293 INFO:teuthology.orchestra.run.smithi138.stdout:(137/177): python3-jaraco-8.2.1-3.el9.noarch.rp 186 kB/s | 11 kB 00:00 2024-01-21T22:18:01.319 INFO:teuthology.orchestra.run.smithi138.stdout:(138/177): python3-jaraco-classes-3.2.1-5.el9.n 356 kB/s | 18 kB 00:00 2024-01-21T22:18:01.344 INFO:teuthology.orchestra.run.smithi138.stdout:(139/177): python3-jaraco-collections-3.0.0-8.e 467 kB/s | 23 kB 00:00 2024-01-21T22:18:01.371 INFO:teuthology.orchestra.run.smithi138.stdout:(140/177): python3-jaraco-functools-3.5.0-2.el9 392 kB/s | 19 kB 00:00 2024-01-21T22:18:01.395 INFO:teuthology.orchestra.run.smithi138.stdout:(141/177): python3-jaraco-text-3.2.0-6.el9.noar 396 kB/s | 20 kB 00:00 2024-01-21T22:18:01.453 INFO:teuthology.orchestra.run.smithi138.stdout:(142/177): python3-logutils-0.3.5-21.el9.noarch 805 kB/s | 46 kB 00:00 2024-01-21T22:18:01.512 INFO:teuthology.orchestra.run.smithi138.stdout:(143/177): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-01-21T22:18:01.570 INFO:teuthology.orchestra.run.smithi138.stdout:(144/177): python3-more-itertools-8.12.0-2.el9. 679 kB/s | 79 kB 00:00 2024-01-21T22:18:01.587 INFO:teuthology.orchestra.run.smithi138.stdout:(145/177): python3-natsort-7.1.1-5.el9.noarch.r 777 kB/s | 58 kB 00:00 2024-01-21T22:18:01.637 INFO:teuthology.orchestra.run.smithi138.stdout:(146/177): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-01-21T22:18:01.663 INFO:teuthology.orchestra.run.smithi138.stdout:(147/177): python3-portend-3.1.0-2.el9.noarch.r 220 kB/s | 16 kB 00:00 2024-01-21T22:18:01.697 INFO:teuthology.orchestra.run.smithi138.stdout:(148/177): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-01-21T22:18:01.722 INFO:teuthology.orchestra.run.smithi138.stdout:(149/177): python3-pyperclip-1.8.2-2.el9.noarch 488 kB/s | 28 kB 00:00 2024-01-21T22:18:01.755 INFO:teuthology.orchestra.run.smithi138.stdout:(150/177): python3-repoze-lru-0.7-16.el9.noarch 534 kB/s | 31 kB 00:00 2024-01-21T22:18:01.789 INFO:teuthology.orchestra.run.smithi138.stdout:(151/177): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-01-21T22:18:01.814 INFO:teuthology.orchestra.run.smithi138.stdout:(152/177): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-01-21T22:18:01.848 INFO:teuthology.orchestra.run.smithi138.stdout:(153/177): python3-tempora-5.0.0-2.el9.noarch.r 619 kB/s | 36 kB 00:00 2024-01-21T22:18:01.873 INFO:teuthology.orchestra.run.smithi138.stdout:(154/177): python3-typing-extensions-4.4.0-2.el 881 kB/s | 51 kB 00:00 2024-01-21T22:18:01.915 INFO:teuthology.orchestra.run.smithi138.stdout:(155/177): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-01-21T22:18:01.982 INFO:teuthology.orchestra.run.smithi138.stdout:(156/177): python3-websocket-client-1.2.3-2.el9 830 kB/s | 90 kB 00:00 2024-01-21T22:18:02.033 INFO:teuthology.orchestra.run.smithi138.stdout:(157/177): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-01-21T22:18:02.058 INFO:teuthology.orchestra.run.smithi138.stdout:(158/177): python3-zc-lockfile-2.0-10.el9.noarc 268 kB/s | 20 kB 00:00 2024-01-21T22:18:02.100 INFO:teuthology.orchestra.run.smithi138.stdout:(159/177): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-01-21T22:18:02.200 INFO:teuthology.orchestra.run.smithi138.stdout:(160/177): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-01-21T22:18:02.984 INFO:teuthology.orchestra.run.smithi138.stdout:(161/177): librados2-19.0.0-811.geb1b049f.el9.x 3.8 MB/s | 3.4 MB 00:00 2024-01-21T22:18:03.035 INFO:teuthology.orchestra.run.smithi138.stdout:(162/177): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.3 MB/s | 68 kB 00:00 2024-01-21T22:18:03.171 INFO:teuthology.orchestra.run.smithi138.stdout:(163/177): NetworkManager-1.45.10-1.el9.x86_64. 17 MB/s | 2.2 MB 00:00 2024-01-21T22:18:03.260 INFO:teuthology.orchestra.run.smithi138.stdout:(164/177): librbd1-19.0.0-811.geb1b049f.el9.x86 3.0 MB/s | 3.1 MB 00:01 2024-01-21T22:18:03.343 INFO:teuthology.orchestra.run.smithi138.stdout:(165/177): NetworkManager-libnm-1.45.10-1.el9.x 11 MB/s | 1.8 MB 00:00 2024-01-21T22:18:03.369 INFO:teuthology.orchestra.run.smithi138.stdout:(166/177): NetworkManager-team-1.45.10-1.el9.x8 442 kB/s | 42 kB 00:00 2024-01-21T22:18:03.402 INFO:teuthology.orchestra.run.smithi138.stdout:(167/177): NetworkManager-tui-1.45.10-1.el9.x86 4.2 MB/s | 246 kB 00:00 2024-01-21T22:18:03.427 INFO:teuthology.orchestra.run.smithi138.stdout:(168/177): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 69 kB 00:00 2024-01-21T22:18:03.453 INFO:teuthology.orchestra.run.smithi138.stdout:(169/177): elfutils-debuginfod-client-0.190-2.e 723 kB/s | 35 kB 00:00 2024-01-21T22:18:03.487 INFO:teuthology.orchestra.run.smithi138.stdout:(170/177): elfutils-libelf-0.190-2.el9.x86_64.r 3.2 MB/s | 191 kB 00:00 2024-01-21T22:18:03.520 INFO:teuthology.orchestra.run.smithi138.stdout:(171/177): elfutils-libs-0.190-2.el9.x86_64.rpm 3.9 MB/s | 254 kB 00:00 2024-01-21T22:18:03.567 INFO:teuthology.orchestra.run.smithi138.stdout:(172/177): libacl-2.3.1-4.el9.x86_64.rpm 283 kB/s | 22 kB 00:00 2024-01-21T22:18:03.621 INFO:teuthology.orchestra.run.smithi138.stdout:(173/177): openssl-3.0.7-25.el9.x86_64.rpm 12 MB/s | 1.2 MB 00:00 2024-01-21T22:18:03.705 INFO:teuthology.orchestra.run.smithi138.stdout:(174/177): openssl-libs-3.0.7-25.el9.x86_64.rpm 17 MB/s | 2.1 MB 00:00 2024-01-21T22:18:03.738 INFO:teuthology.orchestra.run.smithi138.stdout:(175/177): xfsdump-3.1.12-4.el9.x86_64.rpm 3.0 MB/s | 321 kB 00:00 2024-01-21T22:18:03.788 INFO:teuthology.orchestra.run.smithi138.stdout:(176/177): xfsprogs-6.3.0-1.el9.x86_64.rpm 13 MB/s | 1.1 MB 00:00 2024-01-21T22:18:23.278 INFO:teuthology.orchestra.run.smithi138.stdout:(177/177): ceph-test-debuginfo-19.0.0-811.geb1b 11 MB/s | 664 MB 01:00 2024-01-21T22:18:23.288 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-01-21T22:18:23.288 INFO:teuthology.orchestra.run.smithi138.stdout:Total 13 MB/s | 2.1 GB 02:40 2024-01-21T22:18:23.381 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-01-21T22:18:23.733 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-01-21T22:18:23.734 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-01-21T22:18:31.638 INFO:teuthology.orchestra.run.smithi152.stdout:(177/177): ceph-test-debuginfo-19.0.0-811.geb1b 5.9 MB/s | 664 MB 01:52 2024-01-21T22:18:31.643 INFO:teuthology.orchestra.run.smithi152.stdout:-------------------------------------------------------------------------------- 2024-01-21T22:18:31.643 INFO:teuthology.orchestra.run.smithi152.stdout:Total 11 MB/s | 2.1 GB 03:08 2024-01-21T22:18:31.674 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction check 2024-01-21T22:18:31.936 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction check succeeded. 2024-01-21T22:18:31.936 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction test 2024-01-21T22:18:37.618 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-01-21T22:18:37.620 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-01-21T22:18:44.944 INFO:teuthology.orchestra.run.smithi152.stdout:Transaction test succeeded. 2024-01-21T22:18:44.945 INFO:teuthology.orchestra.run.smithi152.stdout:Running transaction 2024-01-21T22:18:58.930 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-01-21T22:18:59.082 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 1/193 2024-01-21T22:18:59.138 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 2/193 2024-01-21T22:18:59.217 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/193 2024-01-21T22:18:59.328 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/193 2024-01-21T22:18:59.383 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/193 2024-01-21T22:18:59.415 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 6/193 2024-01-21T22:18:59.507 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 7/193 2024-01-21T22:19:00.058 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 8/193 2024-01-21T22:19:00.147 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:19.0.0-811.geb1b049f.el9.x86_64 9/193 2024-01-21T22:19:00.320 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:19.0.0-811.geb1b049f.el9.x86_64 9/193 2024-01-21T22:19:00.356 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:19.0.0-811.geb1b049f.el9.x86_64 10/193 2024-01-21T22:19:00.480 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:19.0.0-811.geb1b049f.el9.x86_64 10/193 2024-01-21T22:19:00.521 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:19.0.0-811.geb1b049f.el9.x86_64 11/193 2024-01-21T22:19:00.701 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 12/193 2024-01-21T22:19:00.804 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 13/193 2024-01-21T22:19:01.107 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/193 2024-01-21T22:19:01.154 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/193 2024-01-21T22:19:01.179 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephsqlite-2:19.0.0-811.geb1b049f.el9.x86_64 16/193 2024-01-21T22:19:01.325 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephsqlite-2:19.0.0-811.geb1b049f.el9.x86_64 16/193 2024-01-21T22:19:01.372 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:19.0.0-811.geb1b049f.el9.x86_ 17/193 2024-01-21T22:19:01.448 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:19.0.0-811.geb1b049f.el9.x86_ 17/193 2024-01-21T22:19:01.760 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 18/193 2024-01-21T22:19:01.944 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 19/193 2024-01-21T22:19:02.032 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 19/193 2024-01-21T22:19:02.279 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 20/193 2024-01-21T22:19:02.322 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-1:1.45.10-1.el9.x86_64 20/193 2024-01-21T22:19:02.744 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 20/193 2024-01-21T22:19:02.786 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/193 2024-01-21T22:19:02.894 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/193 2024-01-21T22:19:02.998 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 23/193 2024-01-21T22:19:03.039 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 24/193 2024-01-21T22:19:03.108 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 25/193 2024-01-21T22:19:03.162 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 26/193 2024-01-21T22:19:03.280 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 27/193 2024-01-21T22:19:03.328 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 28/193 2024-01-21T22:19:03.388 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 29/193 2024-01-21T22:19:03.428 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/193 2024-01-21T22:19:03.543 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/193 2024-01-21T22:19:03.582 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:19.0.0-811.geb1b049f.el9.x 32/193 2024-01-21T22:19:03.633 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:19.0.0-811.geb1b049f.el9 33/193 2024-01-21T22:19:03.670 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:19.0.0-811.geb1b049f.el9.x86_64 34/193 2024-01-21T22:19:03.774 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/193 2024-01-21T22:19:03.885 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/193 2024-01-21T22:19:04.064 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/193 2024-01-21T22:19:04.135 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/193 2024-01-21T22:19:04.846 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/193 2024-01-21T22:19:04.937 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/193 2024-01-21T22:19:04.986 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 41/193 2024-01-21T22:19:05.035 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 42/193 2024-01-21T22:19:05.061 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:19.0.0-811.geb1b049f.el9.x86_64 43/193 2024-01-21T22:19:05.115 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/193 2024-01-21T22:19:05.304 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/193 2024-01-21T22:19:05.332 INFO:teuthology.orchestra.run.smithi152.stdout: Preparing : 1/1 2024-01-21T22:19:05.375 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 46/193 2024-01-21T22:19:05.432 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 1/193 2024-01-21T22:19:05.504 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 2/193 2024-01-21T22:19:05.522 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : re2-1:20211101-3.el9.x86_64 47/193 2024-01-21T22:19:05.543 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/193 2024-01-21T22:19:05.565 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 48/193 2024-01-21T22:19:05.596 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/193 2024-01-21T22:19:05.630 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/193 2024-01-21T22:19:05.631 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/193 2024-01-21T22:19:05.664 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 6/193 2024-01-21T22:19:05.707 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 7/193 2024-01-21T22:19:05.717 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/193 2024-01-21T22:19:05.787 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/193 2024-01-21T22:19:05.822 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/193 2024-01-21T22:19:05.890 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/193 2024-01-21T22:19:05.931 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/193 2024-01-21T22:19:05.992 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/193 2024-01-21T22:19:06.032 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 8/193 2024-01-21T22:19:06.052 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/193 2024-01-21T22:19:06.082 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librados2-2:19.0.0-811.geb1b049f.el9.x86_64 9/193 2024-01-21T22:19:06.204 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-2:19.0.0-811.geb1b049f.el9.x86_64 9/193 2024-01-21T22:19:06.233 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs2-2:19.0.0-811.geb1b049f.el9.x86_64 10/193 2024-01-21T22:19:06.242 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/193 2024-01-21T22:19:06.304 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/193 2024-01-21T22:19:06.318 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephfs2-2:19.0.0-811.geb1b049f.el9.x86_64 10/193 2024-01-21T22:19:06.344 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/193 2024-01-21T22:19:06.346 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rados-2:19.0.0-811.geb1b049f.el9.x86_64 11/193 2024-01-21T22:19:06.404 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/193 2024-01-21T22:19:06.425 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 12/193 2024-01-21T22:19:06.513 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 13/193 2024-01-21T22:19:06.545 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/193 2024-01-21T22:19:06.610 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-google-auth-1:2.26.2-1.el9.noarch 62/193 2024-01-21T22:19:06.661 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/193 2024-01-21T22:19:06.685 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/193 2024-01-21T22:19:06.693 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/193 2024-01-21T22:19:06.730 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/193 2024-01-21T22:19:06.758 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephsqlite-2:19.0.0-811.geb1b049f.el9.x86_64 16/193 2024-01-21T22:19:06.844 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 65/193 2024-01-21T22:19:06.849 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libcephsqlite-2:19.0.0-811.geb1b049f.el9.x86_64 16/193 2024-01-21T22:19:06.873 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libradosstriper1-2:19.0.0-811.geb1b049f.el9.x86_ 17/193 2024-01-21T22:19:06.906 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 66/193 2024-01-21T22:19:06.934 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: libradosstriper1-2:19.0.0-811.geb1b049f.el9.x86_ 17/193 2024-01-21T22:19:06.976 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/193 2024-01-21T22:19:07.037 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 18/193 2024-01-21T22:19:07.063 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/193 2024-01-21T22:19:07.120 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 19/193 2024-01-21T22:19:07.179 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 19/193 2024-01-21T22:19:07.253 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 20/193 2024-01-21T22:19:07.289 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : NetworkManager-1:1.45.10-1.el9.x86_64 20/193 2024-01-21T22:19:07.322 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/193 2024-01-21T22:19:07.410 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 70/193 2024-01-21T22:19:07.561 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: NetworkManager-1:1.45.10-1.el9.x86_64 20/193 2024-01-21T22:19:07.599 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/193 2024-01-21T22:19:07.601 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/193 2024-01-21T22:19:07.660 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 72/193 2024-01-21T22:19:07.671 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/193 2024-01-21T22:19:07.701 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 73/193 2024-01-21T22:19:07.724 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 23/193 2024-01-21T22:19:07.751 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 74/193 2024-01-21T22:19:07.755 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 24/193 2024-01-21T22:19:07.791 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 25/193 2024-01-21T22:19:07.823 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : jq-1.6-16.el9.x86_64 75/193 2024-01-21T22:19:07.830 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 26/193 2024-01-21T22:19:07.869 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.9-15.el9.x86_64 76/193 2024-01-21T22:19:07.896 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 27/193 2024-01-21T22:19:07.922 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 77/193 2024-01-21T22:19:07.929 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 28/193 2024-01-21T22:19:07.972 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 29/193 2024-01-21T22:19:07.982 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 78/193 2024-01-21T22:19:08.000 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lua-5.4.4-4.el9.x86_64 30/193 2024-01-21T22:19:08.061 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : uuid-1.6.2-55.el9.x86_64 79/193 2024-01-21T22:19:08.091 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 31/193 2024-01-21T22:19:08.125 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-common-2:19.0.0-811.geb1b049f.el9.x 32/193 2024-01-21T22:19:08.125 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 80/193 2024-01-21T22:19:08.160 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ceph-argparse-2:19.0.0-811.geb1b049f.el9 33/193 2024-01-21T22:19:08.169 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 81/193 2024-01-21T22:19:08.188 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cephfs-2:19.0.0-811.geb1b049f.el9.x86_64 34/193 2024-01-21T22:19:08.258 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/193 2024-01-21T22:19:08.341 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 82/193 2024-01-21T22:19:08.351 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/193 2024-01-21T22:19:08.381 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 83/193 2024-01-21T22:19:08.416 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 84/193 2024-01-21T22:19:08.478 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 85/193 2024-01-21T22:19:08.511 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 86/193 2024-01-21T22:19:08.512 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/193 2024-01-21T22:19:08.563 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/193 2024-01-21T22:19:08.644 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 87/193 2024-01-21T22:19:08.701 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 88/193 2024-01-21T22:19:08.846 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 89/193 2024-01-21T22:19:08.909 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 90/193 2024-01-21T22:19:09.095 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/193 2024-01-21T22:19:09.184 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/193 2024-01-21T22:19:09.214 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 41/193 2024-01-21T22:19:09.255 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 42/193 2024-01-21T22:19:09.293 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librados-devel-2:19.0.0-811.geb1b049f.el9.x86_64 43/193 2024-01-21T22:19:09.334 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/193 2024-01-21T22:19:09.474 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/193 2024-01-21T22:19:09.526 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 46/193 2024-01-21T22:19:09.634 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : re2-1:20211101-3.el9.x86_64 47/193 2024-01-21T22:19:09.672 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 48/193 2024-01-21T22:19:09.727 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/193 2024-01-21T22:19:09.800 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/193 2024-01-21T22:19:09.830 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/193 2024-01-21T22:19:09.863 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/193 2024-01-21T22:19:09.867 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/193 2024-01-21T22:19:09.914 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/193 2024-01-21T22:19:09.925 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 92/193 2024-01-21T22:19:09.949 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/193 2024-01-21T22:19:09.959 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 93/193 2024-01-21T22:19:09.993 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/193 2024-01-21T22:19:10.019 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 94/193 2024-01-21T22:19:10.034 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/193 2024-01-21T22:19:10.142 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 95/193 2024-01-21T22:19:10.171 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/193 2024-01-21T22:19:10.177 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 96/193 2024-01-21T22:19:10.223 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/193 2024-01-21T22:19:10.253 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/193 2024-01-21T22:19:10.294 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/193 2024-01-21T22:19:10.390 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/193 2024-01-21T22:19:10.427 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-google-auth-1:2.26.2-1.el9.noarch 62/193 2024-01-21T22:19:10.453 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-0.3.21-2.el9.x86_64 97/193 2024-01-21T22:19:10.468 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/193 2024-01-21T22:19:10.490 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 98/193 2024-01-21T22:19:10.493 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/193 2024-01-21T22:19:10.589 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 99/193 2024-01-21T22:19:10.590 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 65/193 2024-01-21T22:19:10.637 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 66/193 2024-01-21T22:19:10.686 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/193 2024-01-21T22:19:10.741 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/193 2024-01-21T22:19:10.928 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/193 2024-01-21T22:19:10.982 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 70/193 2024-01-21T22:19:11.115 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/193 2024-01-21T22:19:11.156 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 72/193 2024-01-21T22:19:11.191 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 73/193 2024-01-21T22:19:11.220 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 74/193 2024-01-21T22:19:11.256 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : jq-1.6-16.el9.x86_64 75/193 2024-01-21T22:19:11.286 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : fuse-2.9.9-15.el9.x86_64 76/193 2024-01-21T22:19:11.314 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 77/193 2024-01-21T22:19:11.357 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 78/193 2024-01-21T22:19:11.396 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : uuid-1.6.2-55.el9.x86_64 79/193 2024-01-21T22:19:11.427 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 100/193 2024-01-21T22:19:11.427 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 80/193 2024-01-21T22:19:11.457 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 81/193 2024-01-21T22:19:11.534 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 82/193 2024-01-21T22:19:11.553 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 101/193 2024-01-21T22:19:11.567 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 83/193 2024-01-21T22:19:11.600 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 84/193 2024-01-21T22:19:11.642 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 85/193 2024-01-21T22:19:11.711 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 86/193 2024-01-21T22:19:11.800 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 87/193 2024-01-21T22:19:11.837 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 88/193 2024-01-21T22:19:11.941 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 89/193 2024-01-21T22:19:11.997 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 90/193 2024-01-21T22:19:12.750 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/193 2024-01-21T22:19:12.801 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 92/193 2024-01-21T22:19:12.835 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 93/193 2024-01-21T22:19:12.883 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 94/193 2024-01-21T22:19:12.973 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 95/193 2024-01-21T22:19:13.019 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 96/193 2024-01-21T22:19:13.106 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 102/193 2024-01-21T22:19:13.199 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-0.3.21-2.el9.x86_64 97/193 2024-01-21T22:19:13.210 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 103/193 2024-01-21T22:19:13.225 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 98/193 2024-01-21T22:19:13.249 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 104/193 2024-01-21T22:19:13.291 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 105/193 2024-01-21T22:19:13.304 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 99/193 2024-01-21T22:19:13.665 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 106/193 2024-01-21T22:19:13.706 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:19.0.0-811.geb1b049f.el9.x86_64 107/193 2024-01-21T22:19:13.818 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:19.0.0-811.geb1b049f.el9.x86_64 107/193 2024-01-21T22:19:13.862 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:19.0.0-811.geb1b049f.el9.x86_64 108/193 2024-01-21T22:19:13.920 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 109/193 2024-01-21T22:19:13.960 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 100/193 2024-01-21T22:19:14.082 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 101/193 2024-01-21T22:19:14.410 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 110/193 2024-01-21T22:19:14.443 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:19.0.0-811.geb1b049f.el9.x86_64 111/193 2024-01-21T22:19:14.526 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:19.0.0-811.geb1b049f.el9.x86_64 111/193 2024-01-21T22:19:14.562 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:19.0.0-811.geb1b049f.el9.x86_64 112/193 2024-01-21T22:19:15.330 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 102/193 2024-01-21T22:19:15.412 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 103/193 2024-01-21T22:19:15.439 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 104/193 2024-01-21T22:19:15.472 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 105/193 2024-01-21T22:19:15.780 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 106/193 2024-01-21T22:19:15.810 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : librbd1-2:19.0.0-811.geb1b049f.el9.x86_64 107/193 2024-01-21T22:19:15.900 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-2:19.0.0-811.geb1b049f.el9.x86_64 107/193 2024-01-21T22:19:15.931 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rbd-2:19.0.0-811.geb1b049f.el9.x86_64 108/193 2024-01-21T22:19:15.976 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 109/193 2024-01-21T22:19:16.405 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 110/193 2024-01-21T22:19:16.446 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librgw2-2:19.0.0-811.geb1b049f.el9.x86_64 111/193 2024-01-21T22:19:16.517 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librgw2-2:19.0.0-811.geb1b049f.el9.x86_64 111/193 2024-01-21T22:19:16.556 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rgw-2:19.0.0-811.geb1b049f.el9.x86_64 112/193 2024-01-21T22:19:17.130 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.0.0-811.geb1b049f.el9.x86_64 113/193 2024-01-21T22:19:17.177 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:19.0.0-811.geb1b049f.el9.x86_64 113/193 2024-01-21T22:19:17.787 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.0.0-811.geb1b049f.el9.x86_64 113/193 2024-01-21T22:19:17.856 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:19.0.0-811.geb1b049f.el9.x86_64 114/193 2024-01-21T22:19:18.025 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:19.0.0-811.geb1b049f.el9.x86_64 114/193 2024-01-21T22:19:18.025 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-01-21T22:19:18.025 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-01-21T22:19:18.025 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:19:18.074 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:19.0.0-811.geb1b049f.el9.x86_64 115/193 2024-01-21T22:19:18.609 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:19.0.0-811.geb1b049f.el9.x86_64 113/193 2024-01-21T22:19:18.644 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-common-2:19.0.0-811.geb1b049f.el9.x86_64 113/193 2024-01-21T22:19:19.162 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-common-2:19.0.0-811.geb1b049f.el9.x86_64 113/193 2024-01-21T22:19:19.211 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-base-2:19.0.0-811.geb1b049f.el9.x86_64 114/193 2024-01-21T22:19:19.301 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-base-2:19.0.0-811.geb1b049f.el9.x86_64 114/193 2024-01-21T22:19:19.301 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-01-21T22:19:19.301 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-01-21T22:19:19.301 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:19:19.337 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-selinux-2:19.0.0-811.geb1b049f.el9.x86_64 115/193 2024-01-21T22:19:31.730 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-selinux-2:19.0.0-811.geb1b049f.el9.x86_64 115/193 2024-01-21T22:19:31.730 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /sys 2024-01-21T22:19:31.730 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /proc 2024-01-21T22:19:31.730 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /mnt 2024-01-21T22:19:31.730 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /var/tmp 2024-01-21T22:19:31.730 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /home 2024-01-21T22:19:31.730 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /root 2024-01-21T22:19:31.730 INFO:teuthology.orchestra.run.smithi152.stdout:skipping the directory /tmp 2024-01-21T22:19:31.730 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:19:31.888 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-volume-2:19.0.0-811.geb1b049f.el9.noarch 116/193 2024-01-21T22:19:31.902 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-volume-2:19.0.0-811.geb1b049f.el9.noarch 116/193 2024-01-21T22:19:31.902 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-21T22:19:31.902 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-01-21T22:19:31.902 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:19:31.939 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:19.0.0-811.geb1b049f.el9.x86_64 115/193 2024-01-21T22:19:31.939 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2024-01-21T22:19:31.939 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2024-01-21T22:19:31.939 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2024-01-21T22:19:31.939 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2024-01-21T22:19:31.939 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2024-01-21T22:19:31.939 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /root 2024-01-21T22:19:31.939 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2024-01-21T22:19:31.939 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:19:32.157 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-volume-2:19.0.0-811.geb1b049f.el9.noarch 116/193 2024-01-21T22:19:32.205 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-volume-2:19.0.0-811.geb1b049f.el9.noarch 116/193 2024-01-21T22:19:32.205 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-21T22:19:32.205 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-01-21T22:19:32.205 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:19:33.382 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-osd-2:19.0.0-811.geb1b049f.el9.x86_64 117/193 2024-01-21T22:19:33.411 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-osd-2:19.0.0-811.geb1b049f.el9.x86_64 117/193 2024-01-21T22:19:33.411 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-21T22:19:33.411 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-01-21T22:19:33.411 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-21T22:19:33.411 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-21T22:19:33.411 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:19:33.625 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mds-2:19.0.0-811.geb1b049f.el9.x86_64 118/193 2024-01-21T22:19:33.654 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mds-2:19.0.0-811.geb1b049f.el9.x86_64 118/193 2024-01-21T22:19:33.654 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-21T22:19:33.654 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-01-21T22:19:33.654 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-21T22:19:33.655 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-21T22:19:33.655 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:19:33.792 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:19.0.0-811.geb1b049f.el9.x86_64 117/193 2024-01-21T22:19:33.869 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:19.0.0-811.geb1b049f.el9.x86_64 117/193 2024-01-21T22:19:33.869 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-21T22:19:33.869 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-01-21T22:19:33.869 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-21T22:19:33.869 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-01-21T22:19:33.869 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:19:34.101 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mon-2:19.0.0-811.geb1b049f.el9.x86_64 119/193 2024-01-21T22:19:34.110 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:19.0.0-811.geb1b049f.el9.x86_64 118/193 2024-01-21T22:19:34.124 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mon-2:19.0.0-811.geb1b049f.el9.x86_64 119/193 2024-01-21T22:19:34.124 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-21T22:19:34.124 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-01-21T22:19:34.124 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-21T22:19:34.124 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-21T22:19:34.124 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:19:34.179 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:19.0.0-811.geb1b049f.el9.x86_64 118/193 2024-01-21T22:19:34.179 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-21T22:19:34.179 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-01-21T22:19:34.179 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-21T22:19:34.179 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-01-21T22:19:34.179 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:19:34.211 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : automake-1.16.2-8.el9.noarch 120/193 2024-01-21T22:19:34.544 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephadm-2:19.0.0-811.geb1b049f.el9.noarch 121/193 2024-01-21T22:19:34.576 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephadm-2:19.0.0-811.geb1b049f.el9.noarch 121/193 2024-01-21T22:19:34.606 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-811.geb1b049f.el 122/193 2024-01-21T22:19:34.668 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-811.geb1b049f.e 123/193 2024-01-21T22:19:34.669 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:19.0.0-811.geb1b049f.el9.x86_64 119/193 2024-01-21T22:19:34.691 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-811.geb1b049f.el9.noar 124/193 2024-01-21T22:19:34.736 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:19.0.0-811.geb1b049f.el9.x86_64 119/193 2024-01-21T22:19:34.737 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-21T22:19:34.737 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-01-21T22:19:34.737 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-21T22:19:34.737 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-01-21T22:19:34.737 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:19:34.863 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : automake-1.16.2-8.el9.noarch 120/193 2024-01-21T22:19:34.915 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-811.geb1b049f.el9.noar 124/193 2024-01-21T22:19:34.942 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-811.geb1b049f.el9.no 125/193 2024-01-21T22:19:35.448 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:19.0.0-811.geb1b049f.el9.noarch 121/193 2024-01-21T22:19:35.481 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:19.0.0-811.geb1b049f.el9.noarch 121/193 2024-01-21T22:19:35.527 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-811.geb1b049f.el 122/193 2024-01-21T22:19:35.623 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-811.geb1b049f.e 123/193 2024-01-21T22:19:35.661 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-811.geb1b049f.el9.noar 124/193 2024-01-21T22:19:35.911 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-811.geb1b049f.el9.no 125/193 2024-01-21T22:19:35.942 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-811.geb1b 126/193 2024-01-21T22:19:35.975 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-811.geb1b049f.el9.noar 124/193 2024-01-21T22:19:36.029 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-811.geb1b049f.el9.no 125/193 2024-01-21T22:19:36.042 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-811.geb1b 126/193 2024-01-21T22:19:36.213 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-811.geb1b049f.el9 127/193 2024-01-21T22:19:36.252 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-2:19.0.0-811.geb1b049f.el9.x86_64 128/193 2024-01-21T22:19:36.274 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-2:19.0.0-811.geb1b049f.el9.x86_64 128/193 2024-01-21T22:19:36.274 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-21T22:19:36.275 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-01-21T22:19:36.275 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-21T22:19:36.275 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-21T22:19:36.275 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:19:36.320 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-rook-2:19.0.0-811.geb1b049f.el9.noarch 129/193 2024-01-21T22:19:36.330 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-811.geb1b049f.el9.noarch 129/193 2024-01-21T22:19:36.376 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-2:19.0.0-811.geb1b049f.el9.x86_64 130/193 2024-01-21T22:19:37.151 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-811.geb1b049f.el9.no 125/193 2024-01-21T22:19:37.157 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libtool-2.4.6-45.el9.x86_64 131/193 2024-01-21T22:19:37.188 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-radosgw-2:19.0.0-811.geb1b049f.el9.x86_64 132/193 2024-01-21T22:19:37.203 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-811.geb1b 126/193 2024-01-21T22:19:37.210 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-811.geb1b049f.el9.x86_64 132/193 2024-01-21T22:19:37.210 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-21T22:19:37.210 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-01-21T22:19:37.210 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-21T22:19:37.210 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-21T22:19:37.210 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:19:37.255 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-811.geb1b04 133/193 2024-01-21T22:19:37.277 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-811.geb1b04 133/193 2024-01-21T22:19:37.277 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-21T22:19:37.277 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-01-21T22:19:37.277 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:19:37.331 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-mirror-2:19.0.0-811.geb1b049f.el9.x86_64 134/193 2024-01-21T22:19:37.348 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-811.geb1b 126/193 2024-01-21T22:19:37.353 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-811.geb1b049f.el9.x86_64 134/193 2024-01-21T22:19:37.353 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-21T22:19:37.353 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-01-21T22:19:37.353 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-21T22:19:37.353 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-21T22:19:37.353 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:19:37.525 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-811.geb1b049f.el9 127/193 2024-01-21T22:19:37.565 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:19.0.0-811.geb1b049f.el9.x86_64 128/193 2024-01-21T22:19:37.635 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:19.0.0-811.geb1b049f.el9.x86_64 128/193 2024-01-21T22:19:37.635 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-21T22:19:37.635 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-01-21T22:19:37.636 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-21T22:19:37.636 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-01-21T22:19:37.636 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:19:37.657 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-mirror-2:19.0.0-811.geb1b049f.el9.x86_64 135/193 2024-01-21T22:19:37.679 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: rbd-mirror-2:19.0.0-811.geb1b049f.el9.x86_64 135/193 2024-01-21T22:19:37.679 INFO:teuthology.orchestra.run.smithi152.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-21T22:19:37.679 INFO:teuthology.orchestra.run.smithi152.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-01-21T22:19:37.679 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-21T22:19:37.680 INFO:teuthology.orchestra.run.smithi152.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-21T22:19:37.680 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:19:37.699 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-rook-2:19.0.0-811.geb1b049f.el9.noarch 129/193 2024-01-21T22:19:37.736 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-811.geb1b049f.el9.noarch 129/193 2024-01-21T22:19:37.832 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:19.0.0-811.geb1b049f.el9.x86_64 130/193 2024-01-21T22:19:38.695 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libtool-2.4.6-45.el9.x86_64 131/193 2024-01-21T22:19:38.727 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:19.0.0-811.geb1b049f.el9.x86_64 132/193 2024-01-21T22:19:38.800 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-811.geb1b049f.el9.x86_64 132/193 2024-01-21T22:19:38.801 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-21T22:19:38.801 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-01-21T22:19:38.801 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-21T22:19:38.801 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-01-21T22:19:38.801 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:19:38.861 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-811.geb1b04 133/193 2024-01-21T22:19:38.927 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-811.geb1b04 133/193 2024-01-21T22:19:38.927 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-21T22:19:38.927 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-01-21T22:19:38.928 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:19:39.004 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-mirror-2:19.0.0-811.geb1b049f.el9.x86_64 134/193 2024-01-21T22:19:39.077 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-811.geb1b049f.el9.x86_64 134/193 2024-01-21T22:19:39.077 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-21T22:19:39.078 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-01-21T22:19:39.078 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-21T22:19:39.078 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-01-21T22:19:39.078 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:19:39.471 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:19.0.0-811.geb1b049f.el9.x86_64 135/193 2024-01-21T22:19:39.541 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:19.0.0-811.geb1b049f.el9.x86_64 135/193 2024-01-21T22:19:39.541 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-01-21T22:19:39.541 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-01-21T22:19:39.541 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-21T22:19:39.541 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-01-21T22:19:39.542 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:19:41.977 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-test-2:19.0.0-811.geb1b049f.el9.x86_64 136/193 2024-01-21T22:19:42.017 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-fuse-2:19.0.0-811.geb1b049f.el9.x86_64 137/193 2024-01-21T22:19:42.058 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-nbd-2:19.0.0-811.geb1b049f.el9.x86_64 138/193 2024-01-21T22:19:42.090 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 139/193 2024-01-21T22:19:42.118 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 140/193 2024-01-21T22:19:42.305 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-shell-2:19.0.0-811.geb1b049f.el9.x86_64 141/193 2024-01-21T22:19:42.442 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 142/193 2024-01-21T22:19:42.473 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-fuse-2:19.0.0-811.geb1b049f.el9.x86_64 143/193 2024-01-21T22:19:42.497 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs-devel-2:19.0.0-811.geb1b049f.el9.x86_6 144/193 2024-01-21T22:19:42.529 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 145/193 2024-01-21T22:19:42.559 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : NetworkManager-team-1:1.45.10-1.el9.x86_64 146/193 2024-01-21T22:19:42.630 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : NetworkManager-tui-1:1.45.10-1.el9.x86_64 147/193 2024-01-21T22:19:42.680 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 148/193 2024-01-21T22:19:42.740 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 149/193 2024-01-21T22:19:43.287 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 150/193 2024-01-21T22:19:43.321 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 150/193 2024-01-21T22:19:43.348 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 150/193 2024-01-21T22:19:43.394 INFO:teuthology.orchestra.run.smithi152.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 151/193 2024-01-21T22:19:44.212 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:19.0.0-811.geb1b049f.el9.x86_64 136/193 2024-01-21T22:19:44.228 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : cephfs-top-2:19.0.0-811.geb1b049f.el9.noarch 152/193 2024-01-21T22:19:44.268 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:19.0.0-811.geb1b049f.el9.x86_64 137/193 2024-01-21T22:19:44.324 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:19.0.0-811.geb1b049f.el9.x86_64 138/193 2024-01-21T22:19:44.361 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 139/193 2024-01-21T22:19:44.429 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 140/193 2024-01-21T22:19:44.730 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-shell-2:19.0.0-811.geb1b049f.el9.x86_64 141/193 2024-01-21T22:19:44.872 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 142/193 2024-01-21T22:19:44.917 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:19.0.0-811.geb1b049f.el9.x86_64 143/193 2024-01-21T22:19:44.960 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:19.0.0-811.geb1b049f.el9.x86_6 144/193 2024-01-21T22:19:44.999 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 145/193 2024-01-21T22:19:45.046 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-team-1:1.45.10-1.el9.x86_64 146/193 2024-01-21T22:19:45.163 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : NetworkManager-tui-1:1.45.10-1.el9.x86_64 147/193 2024-01-21T22:19:45.270 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 148/193 2024-01-21T22:19:45.313 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 149/193 2024-01-21T22:19:46.202 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 150/193 2024-01-21T22:19:46.253 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 150/193 2024-01-21T22:19:46.349 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 150/193 2024-01-21T22:19:46.415 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 151/193 2024-01-21T22:19:47.786 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-top-2:19.0.0-811.geb1b049f.el9.noarch 152/193 2024-01-21T22:19:51.198 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 153/193 2024-01-21T22:19:55.216 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 153/193 2024-01-21T22:20:30.456 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-base-debuginfo-2:19.0.0-811.geb1b049f.el9.x 154/193 2024-01-21T22:20:31.353 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-common-debuginfo-2:19.0.0-811.geb1b049f.el9 155/193 2024-01-21T22:20:31.786 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-fuse-debuginfo-2:19.0.0-811.geb1b049f.el9.x 156/193 2024-01-21T22:20:36.435 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:19.0.0-8 157/193 2024-01-21T22:20:36.838 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-debuginfo-2:19.0.0-811.geb1b049f.el9.x 154/193 2024-01-21T22:20:37.505 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-debuginfo-2:19.0.0-811.geb1b049f.el9 155/193 2024-01-21T22:20:38.479 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mds-debuginfo-2:19.0.0-811.geb1b049f.el9.x8 158/193 2024-01-21T22:20:38.544 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-debuginfo-2:19.0.0-811.geb1b049f.el9.x 156/193 2024-01-21T22:20:43.153 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:19.0.0-8 157/193 2024-01-21T22:20:45.348 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-debuginfo-2:19.0.0-811.geb1b049f.el9.x8 158/193 2024-01-21T22:20:48.395 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mgr-debuginfo-2:19.0.0-811.geb1b049f.el9.x8 159/193 2024-01-21T22:20:55.835 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-debuginfo-2:19.0.0-811.geb1b049f.el9.x8 159/193 2024-01-21T22:21:16.132 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-mon-debuginfo-2:19.0.0-811.geb1b049f.el9.x8 160/193 2024-01-21T22:21:25.437 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-debuginfo-2:19.0.0-811.geb1b049f.el9.x8 160/193 2024-01-21T22:21:32.531 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-osd-debuginfo-2:19.0.0-811.geb1b049f.el9.x8 161/193 2024-01-21T22:21:42.797 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-debuginfo-2:19.0.0-811.geb1b049f.el9.x8 161/193 2024-01-21T22:22:38.857 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-radosgw-debuginfo-2:19.0.0-811.geb1b049f.el 162/193 2024-01-21T22:22:39.540 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : ceph-test-debuginfo-2:19.0.0-811.geb1b049f.el9.x 163/193 2024-01-21T22:22:39.846 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libcephfs2-debuginfo-2:19.0.0-811.geb1b049f.el9. 164/193 2024-01-21T22:22:44.825 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librados-devel-debuginfo-2:19.0.0-811.geb1b049f. 165/193 2024-01-21T22:22:50.459 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librados2-debuginfo-2:19.0.0-811.geb1b049f.el9.x 166/193 2024-01-21T22:22:50.541 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : librbd1-debuginfo-2:19.0.0-811.geb1b049f.el9.x86 167/193 2024-01-21T22:22:50.640 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-cephfs-debuginfo-2:19.0.0-811.geb1b049f. 168/193 2024-01-21T22:22:50.776 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rados-debuginfo-2:19.0.0-811.geb1b049f.e 169/193 2024-01-21T22:22:50.833 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rbd-debuginfo-2:19.0.0-811.geb1b049f.el9 170/193 2024-01-21T22:22:50.913 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : python3-rgw-debuginfo-2:19.0.0-811.geb1b049f.el9 171/193 2024-01-21T22:22:54.413 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-debuginfo-2:19.0.0-811.geb1b049f.el 162/193 2024-01-21T22:22:55.003 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-debuginfo-2:19.0.0-811.geb1b049f.el9.x 163/193 2024-01-21T22:22:55.043 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-debuginfo-2:19.0.0-811.geb1b049f.el9. 164/193 2024-01-21T22:22:56.483 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-fuse-debuginfo-2:19.0.0-811.geb1b049f.el9.x8 172/193 2024-01-21T22:22:56.634 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-mirror-debuginfo-2:19.0.0-811.geb1b049f.el9. 173/193 2024-01-21T22:22:56.672 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : rbd-nbd-debuginfo-2:19.0.0-811.geb1b049f.el9.x86 174/193 2024-01-21T22:22:56.713 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 175/193 2024-01-21T22:22:56.793 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : flex-2.6.4-9.el9.x86_64 176/193 2024-01-21T22:22:56.826 INFO:teuthology.orchestra.run.smithi152.stdout: Installing : bison-3.7.4-5.el9.x86_64 177/193 2024-01-21T22:22:56.850 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 178/193 2024-01-21T22:22:56.876 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 179/193 2024-01-21T22:22:56.902 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 180/193 2024-01-21T22:22:56.902 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 181/193 2024-01-21T22:22:56.978 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 181/193 2024-01-21T22:22:57.015 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 182/193 2024-01-21T22:22:57.045 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 183/193 2024-01-21T22:22:57.079 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/193 2024-01-21T22:22:57.113 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 185/193 2024-01-21T22:22:57.113 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 186/193 2024-01-21T22:22:57.197 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 186/193 2024-01-21T22:22:57.222 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 187/193 2024-01-21T22:22:57.227 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 188/193 2024-01-21T22:22:57.228 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 188/193 2024-01-21T22:22:57.408 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 188/193 2024-01-21T22:22:57.408 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 189/193 2024-01-21T22:22:57.491 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 189/193 2024-01-21T22:22:57.524 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 190/193 2024-01-21T22:22:57.550 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 191/193 2024-01-21T22:22:57.583 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 192/193 2024-01-21T22:22:57.648 INFO:teuthology.orchestra.run.smithi152.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 193/193 2024-01-21T22:23:00.047 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-debuginfo-2:19.0.0-811.geb1b049f. 165/193 2024-01-21T22:23:07.085 INFO:teuthology.orchestra.run.smithi152.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 193/193 2024-01-21T22:23:07.086 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-2:19.0.0-811.geb1b049f.el9.x86_64 1/193 2024-01-21T22:23:07.086 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-base-2:19.0.0-811.geb1b049f.el9.x86_64 2/193 2024-01-21T22:23:07.086 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-base-debuginfo-2:19.0.0-811.geb1b049f.el9.x 3/193 2024-01-21T22:23:07.086 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-common-2:19.0.0-811.geb1b049f.el9.x86_64 4/193 2024-01-21T22:23:07.086 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-common-debuginfo-2:19.0.0-811.geb1b049f.el9 5/193 2024-01-21T22:23:07.086 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 6/193 2024-01-21T22:23:07.087 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-fuse-2:19.0.0-811.geb1b049f.el9.x86_64 7/193 2024-01-21T22:23:07.087 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-fuse-debuginfo-2:19.0.0-811.geb1b049f.el9.x 8/193 2024-01-21T22:23:07.087 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-811.geb1b04 9/193 2024-01-21T22:23:07.087 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:19.0.0-8 10/193 2024-01-21T22:23:07.087 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mds-2:19.0.0-811.geb1b049f.el9.x86_64 11/193 2024-01-21T22:23:07.087 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mds-debuginfo-2:19.0.0-811.geb1b049f.el9.x8 12/193 2024-01-21T22:23:07.087 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-2:19.0.0-811.geb1b049f.el9.x86_64 13/193 2024-01-21T22:23:07.088 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-debuginfo-2:19.0.0-811.geb1b049f.el9.x8 14/193 2024-01-21T22:23:07.088 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mon-2:19.0.0-811.geb1b049f.el9.x86_64 15/193 2024-01-21T22:23:07.088 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mon-debuginfo-2:19.0.0-811.geb1b049f.el9.x8 16/193 2024-01-21T22:23:07.088 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-osd-2:19.0.0-811.geb1b049f.el9.x86_64 17/193 2024-01-21T22:23:07.088 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-osd-debuginfo-2:19.0.0-811.geb1b049f.el9.x8 18/193 2024-01-21T22:23:07.088 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-radosgw-2:19.0.0-811.geb1b049f.el9.x86_64 19/193 2024-01-21T22:23:07.089 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-radosgw-debuginfo-2:19.0.0-811.geb1b049f.el 20/193 2024-01-21T22:23:07.089 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-selinux-2:19.0.0-811.geb1b049f.el9.x86_64 21/193 2024-01-21T22:23:07.089 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-test-2:19.0.0-811.geb1b049f.el9.x86_64 22/193 2024-01-21T22:23:07.089 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-test-debuginfo-2:19.0.0-811.geb1b049f.el9.x 23/193 2024-01-21T22:23:07.089 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-mirror-2:19.0.0-811.geb1b049f.el9.x86_64 24/193 2024-01-21T22:23:07.089 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-shell-2:19.0.0-811.geb1b049f.el9.x86_64 25/193 2024-01-21T22:23:07.090 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs-devel-2:19.0.0-811.geb1b049f.el9.x86_6 26/193 2024-01-21T22:23:07.090 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs2-2:19.0.0-811.geb1b049f.el9.x86_64 27/193 2024-01-21T22:23:07.090 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephfs2-debuginfo-2:19.0.0-811.geb1b049f.el9. 28/193 2024-01-21T22:23:07.090 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libcephsqlite-2:19.0.0-811.geb1b049f.el9.x86_64 29/193 2024-01-21T22:23:07.090 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados-devel-2:19.0.0-811.geb1b049f.el9.x86_64 30/193 2024-01-21T22:23:07.090 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados-devel-debuginfo-2:19.0.0-811.geb1b049f. 31/193 2024-01-21T22:23:07.091 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-debuginfo-2:19.0.0-811.geb1b049f.el9.x 32/193 2024-01-21T22:23:07.091 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libradosstriper1-2:19.0.0-811.geb1b049f.el9.x86_ 33/193 2024-01-21T22:23:07.091 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-debuginfo-2:19.0.0-811.geb1b049f.el9.x86 34/193 2024-01-21T22:23:07.091 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librgw2-2:19.0.0-811.geb1b049f.el9.x86_64 35/193 2024-01-21T22:23:07.091 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-argparse-2:19.0.0-811.geb1b049f.el9 36/193 2024-01-21T22:23:07.091 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ceph-common-2:19.0.0-811.geb1b049f.el9.x 37/193 2024-01-21T22:23:07.092 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cephfs-2:19.0.0-811.geb1b049f.el9.x86_64 38/193 2024-01-21T22:23:07.092 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cephfs-debuginfo-2:19.0.0-811.geb1b049f. 39/193 2024-01-21T22:23:07.092 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rados-2:19.0.0-811.geb1b049f.el9.x86_64 40/193 2024-01-21T22:23:07.092 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rados-debuginfo-2:19.0.0-811.geb1b049f.e 41/193 2024-01-21T22:23:07.092 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rbd-2:19.0.0-811.geb1b049f.el9.x86_64 42/193 2024-01-21T22:23:07.092 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rbd-debuginfo-2:19.0.0-811.geb1b049f.el9 43/193 2024-01-21T22:23:07.093 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rgw-2:19.0.0-811.geb1b049f.el9.x86_64 44/193 2024-01-21T22:23:07.093 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rgw-debuginfo-2:19.0.0-811.geb1b049f.el9 45/193 2024-01-21T22:23:07.093 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-fuse-2:19.0.0-811.geb1b049f.el9.x86_64 46/193 2024-01-21T22:23:07.093 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-fuse-debuginfo-2:19.0.0-811.geb1b049f.el9.x8 47/193 2024-01-21T22:23:07.093 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-mirror-2:19.0.0-811.geb1b049f.el9.x86_64 48/193 2024-01-21T22:23:07.093 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-mirror-debuginfo-2:19.0.0-811.geb1b049f.el9. 49/193 2024-01-21T22:23:07.094 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-nbd-2:19.0.0-811.geb1b049f.el9.x86_64 50/193 2024-01-21T22:23:07.094 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : rbd-nbd-debuginfo-2:19.0.0-811.geb1b049f.el9.x86 51/193 2024-01-21T22:23:07.094 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-811.geb1b049f.e 52/193 2024-01-21T22:23:07.094 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-811.geb1b049f.el9.noar 53/193 2024-01-21T22:23:07.094 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-811.geb1b049f.el9.no 54/193 2024-01-21T22:23:07.095 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-811.geb1b 55/193 2024-01-21T22:23:07.095 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-811.geb1b049f.el9 56/193 2024-01-21T22:23:07.095 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-mgr-rook-2:19.0.0-811.geb1b049f.el9.noarch 57/193 2024-01-21T22:23:07.095 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-811.geb1b049f.el 58/193 2024-01-21T22:23:07.095 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : ceph-volume-2:19.0.0-811.geb1b049f.el9.noarch 59/193 2024-01-21T22:23:07.095 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephadm-2:19.0.0-811.geb1b049f.el9.noarch 60/193 2024-01-21T22:23:07.096 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : cephfs-top-2:19.0.0-811.geb1b049f.el9.noarch 61/193 2024-01-21T22:23:07.096 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : automake-1.16.2-8.el9.noarch 62/193 2024-01-21T22:23:07.096 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : bison-3.7.4-5.el9.x86_64 63/193 2024-01-21T22:23:07.096 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 64/193 2024-01-21T22:23:07.096 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 65/193 2024-01-21T22:23:07.097 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flex-2.6.4-9.el9.x86_64 66/193 2024-01-21T22:23:07.097 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 67/193 2024-01-21T22:23:07.097 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 68/193 2024-01-21T22:23:07.097 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 69/193 2024-01-21T22:23:07.097 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 70/193 2024-01-21T22:23:07.098 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 71/193 2024-01-21T22:23:07.098 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 72/193 2024-01-21T22:23:07.098 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 73/193 2024-01-21T22:23:07.098 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 74/193 2024-01-21T22:23:07.098 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 75/193 2024-01-21T22:23:07.098 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 76/193 2024-01-21T22:23:07.099 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 77/193 2024-01-21T22:23:07.099 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lua-5.4.4-4.el9.x86_64 78/193 2024-01-21T22:23:07.099 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 79/193 2024-01-21T22:23:07.099 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 80/193 2024-01-21T22:23:07.099 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 81/193 2024-01-21T22:23:07.099 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-13.11-1.el9.x86_64 82/193 2024-01-21T22:23:07.100 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 83/193 2024-01-21T22:23:07.100 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 84/193 2024-01-21T22:23:07.100 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 85/193 2024-01-21T22:23:07.100 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 86/193 2024-01-21T22:23:07.100 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 87/193 2024-01-21T22:23:07.100 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 88/193 2024-01-21T22:23:07.101 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 89/193 2024-01-21T22:23:07.101 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 90/193 2024-01-21T22:23:07.101 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 91/193 2024-01-21T22:23:07.101 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 92/193 2024-01-21T22:23:07.101 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 93/193 2024-01-21T22:23:07.101 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/193 2024-01-21T22:23:07.102 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 95/193 2024-01-21T22:23:07.102 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 96/193 2024-01-21T22:23:07.102 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 97/193 2024-01-21T22:23:07.102 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 98/193 2024-01-21T22:23:07.102 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 99/193 2024-01-21T22:23:07.102 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 100/193 2024-01-21T22:23:07.102 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 101/193 2024-01-21T22:23:07.102 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 102/193 2024-01-21T22:23:07.103 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 103/193 2024-01-21T22:23:07.103 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 104/193 2024-01-21T22:23:07.103 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 105/193 2024-01-21T22:23:07.103 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 106/193 2024-01-21T22:23:07.103 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 107/193 2024-01-21T22:23:07.103 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 108/193 2024-01-21T22:23:07.103 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 109/193 2024-01-21T22:23:07.103 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : jq-1.6-16.el9.x86_64 110/193 2024-01-21T22:23:07.103 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 111/193 2024-01-21T22:23:07.104 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 112/193 2024-01-21T22:23:07.104 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 113/193 2024-01-21T22:23:07.104 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 114/193 2024-01-21T22:23:07.104 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 115/193 2024-01-21T22:23:07.104 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-ply-3.11-14.el9.noarch 116/193 2024-01-21T22:23:07.104 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 117/193 2024-01-21T22:23:07.104 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 118/193 2024-01-21T22:23:07.105 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 119/193 2024-01-21T22:23:07.105 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 120/193 2024-01-21T22:23:07.105 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 121/193 2024-01-21T22:23:07.105 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 122/193 2024-01-21T22:23:07.105 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 123/193 2024-01-21T22:23:07.105 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 124/193 2024-01-21T22:23:07.105 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 125/193 2024-01-21T22:23:07.106 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 126/193 2024-01-21T22:23:07.106 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 127/193 2024-01-21T22:23:07.106 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 128/193 2024-01-21T22:23:07.106 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 129/193 2024-01-21T22:23:07.106 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 130/193 2024-01-21T22:23:07.106 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 131/193 2024-01-21T22:23:07.107 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 132/193 2024-01-21T22:23:07.107 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 133/193 2024-01-21T22:23:07.107 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 134/193 2024-01-21T22:23:07.107 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 135/193 2024-01-21T22:23:07.107 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 136/193 2024-01-21T22:23:07.107 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-google-auth-1:2.26.2-1.el9.noarch 137/193 2024-01-21T22:23:07.107 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 138/193 2024-01-21T22:23:07.108 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 139/193 2024-01-21T22:23:07.108 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 140/193 2024-01-21T22:23:07.108 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 141/193 2024-01-21T22:23:07.108 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 142/193 2024-01-21T22:23:07.108 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 143/193 2024-01-21T22:23:07.108 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 144/193 2024-01-21T22:23:07.108 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 145/193 2024-01-21T22:23:07.108 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 146/193 2024-01-21T22:23:07.109 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 147/193 2024-01-21T22:23:07.109 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 148/193 2024-01-21T22:23:07.109 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 149/193 2024-01-21T22:23:07.109 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 150/193 2024-01-21T22:23:07.109 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 151/193 2024-01-21T22:23:07.109 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 152/193 2024-01-21T22:23:07.109 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 153/193 2024-01-21T22:23:07.109 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 154/193 2024-01-21T22:23:07.109 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 155/193 2024-01-21T22:23:07.109 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 156/193 2024-01-21T22:23:07.109 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 157/193 2024-01-21T22:23:07.109 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 158/193 2024-01-21T22:23:07.109 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 159/193 2024-01-21T22:23:07.110 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : re2-1:20211101-3.el9.x86_64 160/193 2024-01-21T22:23:07.110 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 161/193 2024-01-21T22:23:07.110 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-2:19.0.0-811.geb1b049f.el9.x86_64 162/193 2024-01-21T22:23:07.110 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 163/193 2024-01-21T22:23:07.110 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-2:19.0.0-811.geb1b049f.el9.x86_64 164/193 2024-01-21T22:23:07.110 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 165/193 2024-01-21T22:23:07.110 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 166/193 2024-01-21T22:23:07.110 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 167/193 2024-01-21T22:23:07.110 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-1:1.45.10-1.el9.x86_64 168/193 2024-01-21T22:23:07.110 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 169/193 2024-01-21T22:23:07.110 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 170/193 2024-01-21T22:23:07.110 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 171/193 2024-01-21T22:23:07.110 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-team-1:1.45.10-1.el9.x86_64 172/193 2024-01-21T22:23:07.110 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 173/193 2024-01-21T22:23:07.111 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-tui-1:1.45.10-1.el9.x86_64 174/193 2024-01-21T22:23:07.111 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/193 2024-01-21T22:23:07.111 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : acl-2.3.1-4.el9.x86_64 176/193 2024-01-21T22:23:07.111 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : acl-2.3.1-3.el9.x86_64 177/193 2024-01-21T22:23:07.111 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 178/193 2024-01-21T22:23:07.111 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 179/193 2024-01-21T22:23:07.111 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 180/193 2024-01-21T22:23:07.111 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 181/193 2024-01-21T22:23:07.111 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 182/193 2024-01-21T22:23:07.111 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 183/193 2024-01-21T22:23:07.111 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 184/193 2024-01-21T22:23:07.111 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 185/193 2024-01-21T22:23:07.111 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 186/193 2024-01-21T22:23:07.111 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 187/193 2024-01-21T22:23:07.111 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 188/193 2024-01-21T22:23:07.112 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 189/193 2024-01-21T22:23:07.112 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 190/193 2024-01-21T22:23:07.112 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 191/193 2024-01-21T22:23:07.112 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 192/193 2024-01-21T22:23:07.389 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados2-debuginfo-2:19.0.0-811.geb1b049f.el9.x 166/193 2024-01-21T22:23:07.492 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librbd1-debuginfo-2:19.0.0-811.geb1b049f.el9.x86 167/193 2024-01-21T22:23:07.591 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-debuginfo-2:19.0.0-811.geb1b049f. 168/193 2024-01-21T22:23:07.729 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-debuginfo-2:19.0.0-811.geb1b049f.e 169/193 2024-01-21T22:23:07.799 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-debuginfo-2:19.0.0-811.geb1b049f.el9 170/193 2024-01-21T22:23:07.992 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-debuginfo-2:19.0.0-811.geb1b049f.el9 171/193 2024-01-21T22:23:09.781 INFO:teuthology.orchestra.run.smithi152.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 193/193 2024-01-21T22:23:09.782 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:09.782 INFO:teuthology.orchestra.run.smithi152.stdout:Upgraded: 2024-01-21T22:23:09.782 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-1:1.45.10-1.el9.x86_64 2024-01-21T22:23:09.782 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 2024-01-21T22:23:09.782 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-team-1:1.45.10-1.el9.x86_64 2024-01-21T22:23:09.782 INFO:teuthology.orchestra.run.smithi152.stdout: NetworkManager-tui-1:1.45.10-1.el9.x86_64 2024-01-21T22:23:09.782 INFO:teuthology.orchestra.run.smithi152.stdout: acl-2.3.1-4.el9.x86_64 2024-01-21T22:23:09.782 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-01-21T22:23:09.782 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-01-21T22:23:09.782 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-01-21T22:23:09.782 INFO:teuthology.orchestra.run.smithi152.stdout: libacl-2.3.1-4.el9.x86_64 2024-01-21T22:23:09.782 INFO:teuthology.orchestra.run.smithi152.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-01-21T22:23:09.782 INFO:teuthology.orchestra.run.smithi152.stdout: librados2-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.782 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.783 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-01-21T22:23:09.783 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-01-21T22:23:09.783 INFO:teuthology.orchestra.run.smithi152.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-01-21T22:23:09.783 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-01-21T22:23:09.783 INFO:teuthology.orchestra.run.smithi152.stdout:Installed: 2024-01-21T22:23:09.783 INFO:teuthology.orchestra.run.smithi152.stdout: automake-1.16.2-8.el9.noarch 2024-01-21T22:23:09.783 INFO:teuthology.orchestra.run.smithi152.stdout: bison-3.7.4-5.el9.x86_64 2024-01-21T22:23:09.783 INFO:teuthology.orchestra.run.smithi152.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-01-21T22:23:09.783 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.783 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.783 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-base-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.783 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.783 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-common-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.783 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.784 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.784 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-fuse-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.784 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-grafana-dashboards-2:19.0.0-811.geb1b049f.el9.noarch 2024-01-21T22:23:09.784 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.784 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-immutable-object-cache-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.784 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.784 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mds-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.784 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.784 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-cephadm-2:19.0.0-811.geb1b049f.el9.noarch 2024-01-21T22:23:09.784 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-dashboard-2:19.0.0-811.geb1b049f.el9.noarch 2024-01-21T22:23:09.784 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.784 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-diskprediction-local-2:19.0.0-811.geb1b049f.el9.noarch 2024-01-21T22:23:09.784 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-modules-core-2:19.0.0-811.geb1b049f.el9.noarch 2024-01-21T22:23:09.785 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mgr-rook-2:19.0.0-811.geb1b049f.el9.noarch 2024-01-21T22:23:09.785 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.785 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-mon-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.785 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.785 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-osd-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.785 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-prometheus-alerts-2:19.0.0-811.geb1b049f.el9.noarch 2024-01-21T22:23:09.785 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.785 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-radosgw-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.785 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-selinux-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.785 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.785 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-test-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.785 INFO:teuthology.orchestra.run.smithi152.stdout: ceph-volume-2:19.0.0-811.geb1b049f.el9.noarch 2024-01-21T22:23:09.785 INFO:teuthology.orchestra.run.smithi152.stdout: cephadm-2:19.0.0-811.geb1b049f.el9.noarch 2024-01-21T22:23:09.786 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-mirror-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.786 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-shell-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.786 INFO:teuthology.orchestra.run.smithi152.stdout: cephfs-top-2:19.0.0-811.geb1b049f.el9.noarch 2024-01-21T22:23:09.786 INFO:teuthology.orchestra.run.smithi152.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-01-21T22:23:09.786 INFO:teuthology.orchestra.run.smithi152.stdout: flex-2.6.4-9.el9.x86_64 2024-01-21T22:23:09.786 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-01-21T22:23:09.786 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-01-21T22:23:09.786 INFO:teuthology.orchestra.run.smithi152.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-01-21T22:23:09.786 INFO:teuthology.orchestra.run.smithi152.stdout: fuse-2.9.9-15.el9.x86_64 2024-01-21T22:23:09.786 INFO:teuthology.orchestra.run.smithi152.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-01-21T22:23:09.786 INFO:teuthology.orchestra.run.smithi152.stdout: jq-1.6-16.el9.x86_64 2024-01-21T22:23:09.786 INFO:teuthology.orchestra.run.smithi152.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-01-21T22:23:09.786 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-9.0.0-10.el9.x86_64 2024-01-21T22:23:09.786 INFO:teuthology.orchestra.run.smithi152.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-01-21T22:23:09.787 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs-devel-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.787 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.787 INFO:teuthology.orchestra.run.smithi152.stdout: libcephfs2-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.787 INFO:teuthology.orchestra.run.smithi152.stdout: libcephsqlite-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.787 INFO:teuthology.orchestra.run.smithi152.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-01-21T22:23:09.787 INFO:teuthology.orchestra.run.smithi152.stdout: liboath-2.6.7-2.el9.x86_64 2024-01-21T22:23:09.787 INFO:teuthology.orchestra.run.smithi152.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-01-21T22:23:09.787 INFO:teuthology.orchestra.run.smithi152.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-01-21T22:23:09.787 INFO:teuthology.orchestra.run.smithi152.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-01-21T22:23:09.787 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.787 INFO:teuthology.orchestra.run.smithi152.stdout: librados-devel-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.787 INFO:teuthology.orchestra.run.smithi152.stdout: librados2-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.787 INFO:teuthology.orchestra.run.smithi152.stdout: libradosstriper1-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.787 INFO:teuthology.orchestra.run.smithi152.stdout: librbd1-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.788 INFO:teuthology.orchestra.run.smithi152.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-01-21T22:23:09.788 INFO:teuthology.orchestra.run.smithi152.stdout: librgw2-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.788 INFO:teuthology.orchestra.run.smithi152.stdout: libtool-2.4.6-45.el9.x86_64 2024-01-21T22:23:09.788 INFO:teuthology.orchestra.run.smithi152.stdout: libunwind-1.6.2-1.el9.x86_64 2024-01-21T22:23:09.788 INFO:teuthology.orchestra.run.smithi152.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-01-21T22:23:09.788 INFO:teuthology.orchestra.run.smithi152.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-01-21T22:23:09.788 INFO:teuthology.orchestra.run.smithi152.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-01-21T22:23:09.788 INFO:teuthology.orchestra.run.smithi152.stdout: lua-5.4.4-4.el9.x86_64 2024-01-21T22:23:09.788 INFO:teuthology.orchestra.run.smithi152.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-01-21T22:23:09.788 INFO:teuthology.orchestra.run.smithi152.stdout: luarocks-3.9.2-1.el9.noarch 2024-01-21T22:23:09.788 INFO:teuthology.orchestra.run.smithi152.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-01-21T22:23:09.788 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-0.3.21-2.el9.x86_64 2024-01-21T22:23:09.788 INFO:teuthology.orchestra.run.smithi152.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-01-21T22:23:09.789 INFO:teuthology.orchestra.run.smithi152.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-01-21T22:23:09.789 INFO:teuthology.orchestra.run.smithi152.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-01-21T22:23:09.789 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-13.11-1.el9.x86_64 2024-01-21T22:23:09.789 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-01-21T22:23:09.789 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-01-21T22:23:09.789 INFO:teuthology.orchestra.run.smithi152.stdout: postgresql-server-13.11-1.el9.x86_64 2024-01-21T22:23:09.789 INFO:teuthology.orchestra.run.smithi152.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-01-21T22:23:09.789 INFO:teuthology.orchestra.run.smithi152.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-01-21T22:23:09.789 INFO:teuthology.orchestra.run.smithi152.stdout: python3-babel-2.9.1-2.el9.noarch 2024-01-21T22:23:09.789 INFO:teuthology.orchestra.run.smithi152.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-01-21T22:23:09.789 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-01-21T22:23:09.789 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-argparse-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.789 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ceph-common-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.789 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.790 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cephfs-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.790 INFO:teuthology.orchestra.run.smithi152.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-01-21T22:23:09.790 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-01-21T22:23:09.790 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-01-21T22:23:09.790 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-01-21T22:23:09.790 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-01-21T22:23:09.790 INFO:teuthology.orchestra.run.smithi152.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-01-21T22:23:09.790 INFO:teuthology.orchestra.run.smithi152.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-01-21T22:23:09.790 INFO:teuthology.orchestra.run.smithi152.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-01-21T22:23:09.790 INFO:teuthology.orchestra.run.smithi152.stdout: python3-google-auth-1:2.26.2-1.el9.noarch 2024-01-21T22:23:09.790 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-01-21T22:23:09.790 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-01-21T22:23:09.790 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-01-21T22:23:09.790 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-01-21T22:23:09.790 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-01-21T22:23:09.791 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-01-21T22:23:09.791 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-01-21T22:23:09.791 INFO:teuthology.orchestra.run.smithi152.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-01-21T22:23:09.791 INFO:teuthology.orchestra.run.smithi152.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-01-21T22:23:09.791 INFO:teuthology.orchestra.run.smithi152.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-01-21T22:23:09.791 INFO:teuthology.orchestra.run.smithi152.stdout: python3-mako-1.1.4-6.el9.noarch 2024-01-21T22:23:09.791 INFO:teuthology.orchestra.run.smithi152.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-01-21T22:23:09.791 INFO:teuthology.orchestra.run.smithi152.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-01-21T22:23:09.791 INFO:teuthology.orchestra.run.smithi152.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-01-21T22:23:09.791 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-01-21T22:23:09.791 INFO:teuthology.orchestra.run.smithi152.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-01-21T22:23:09.791 INFO:teuthology.orchestra.run.smithi152.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-01-21T22:23:09.791 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-01-21T22:23:09.792 INFO:teuthology.orchestra.run.smithi152.stdout: python3-ply-3.11-14.el9.noarch 2024-01-21T22:23:09.792 INFO:teuthology.orchestra.run.smithi152.stdout: python3-portend-3.1.0-2.el9.noarch 2024-01-21T22:23:09.792 INFO:teuthology.orchestra.run.smithi152.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-01-21T22:23:09.792 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-01-21T22:23:09.792 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-01-21T22:23:09.792 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-01-21T22:23:09.792 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pycparser-2.20-6.el9.noarch 2024-01-21T22:23:09.792 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-01-21T22:23:09.792 INFO:teuthology.orchestra.run.smithi152.stdout: python3-pytz-2021.1-5.el9.noarch 2024-01-21T22:23:09.792 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.792 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rados-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.792 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.792 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rbd-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.793 INFO:teuthology.orchestra.run.smithi152.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-01-21T22:23:09.793 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-2.25.1-7.el9.noarch 2024-01-21T22:23:09.793 INFO:teuthology.orchestra.run.smithi152.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-01-21T22:23:09.793 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.793 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rgw-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.793 INFO:teuthology.orchestra.run.smithi152.stdout: python3-routes-2.5.1-5.el9.noarch 2024-01-21T22:23:09.793 INFO:teuthology.orchestra.run.smithi152.stdout: python3-rsa-4.9-2.el9.noarch 2024-01-21T22:23:09.793 INFO:teuthology.orchestra.run.smithi152.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-01-21T22:23:09.793 INFO:teuthology.orchestra.run.smithi152.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-01-21T22:23:09.793 INFO:teuthology.orchestra.run.smithi152.stdout: python3-toml-0.10.2-6.el9.noarch 2024-01-21T22:23:09.793 INFO:teuthology.orchestra.run.smithi152.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-01-21T22:23:09.793 INFO:teuthology.orchestra.run.smithi152.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-01-21T22:23:09.793 INFO:teuthology.orchestra.run.smithi152.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-01-21T22:23:09.793 INFO:teuthology.orchestra.run.smithi152.stdout: python3-webob-1.8.7-6.el9.noarch 2024-01-21T22:23:09.794 INFO:teuthology.orchestra.run.smithi152.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-01-21T22:23:09.794 INFO:teuthology.orchestra.run.smithi152.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-01-21T22:23:09.794 INFO:teuthology.orchestra.run.smithi152.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-01-21T22:23:09.794 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.794 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-fuse-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.794 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.794 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-mirror-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.794 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.794 INFO:teuthology.orchestra.run.smithi152.stdout: rbd-nbd-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:09.794 INFO:teuthology.orchestra.run.smithi152.stdout: re2-1:20211101-3.el9.x86_64 2024-01-21T22:23:09.794 INFO:teuthology.orchestra.run.smithi152.stdout: socat-1.7.4.1-5.el9.x86_64 2024-01-21T22:23:09.794 INFO:teuthology.orchestra.run.smithi152.stdout: thrift-0.15.0-2.el9.x86_64 2024-01-21T22:23:09.794 INFO:teuthology.orchestra.run.smithi152.stdout: uuid-1.6.2-55.el9.x86_64 2024-01-21T22:23:09.794 INFO:teuthology.orchestra.run.smithi152.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-01-21T22:23:09.795 INFO:teuthology.orchestra.run.smithi152.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-01-21T22:23:09.795 INFO:teuthology.orchestra.run.smithi152.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-01-21T22:23:09.795 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:09.795 INFO:teuthology.orchestra.run.smithi152.stdout:Complete! 2024-01-21T22:23:10.457 DEBUG:teuthology.parallel:result is None 2024-01-21T22:23:16.023 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-debuginfo-2:19.0.0-811.geb1b049f.el9.x8 172/193 2024-01-21T22:23:16.203 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-debuginfo-2:19.0.0-811.geb1b049f.el9. 173/193 2024-01-21T22:23:16.229 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-debuginfo-2:19.0.0-811.geb1b049f.el9.x86 174/193 2024-01-21T22:23:16.282 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 175/193 2024-01-21T22:23:16.395 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flex-2.6.4-9.el9.x86_64 176/193 2024-01-21T22:23:16.427 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : bison-3.7.4-5.el9.x86_64 177/193 2024-01-21T22:23:16.495 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 178/193 2024-01-21T22:23:16.528 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 179/193 2024-01-21T22:23:16.576 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 180/193 2024-01-21T22:23:16.576 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 181/193 2024-01-21T22:23:16.711 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 181/193 2024-01-21T22:23:16.762 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 182/193 2024-01-21T22:23:16.818 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 183/193 2024-01-21T22:23:16.867 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 184/193 2024-01-21T22:23:16.933 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 185/193 2024-01-21T22:23:16.933 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 186/193 2024-01-21T22:23:17.063 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 186/193 2024-01-21T22:23:17.093 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 187/193 2024-01-21T22:23:17.135 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 188/193 2024-01-21T22:23:17.135 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 188/193 2024-01-21T22:23:17.434 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 188/193 2024-01-21T22:23:17.434 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 189/193 2024-01-21T22:23:17.568 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 189/193 2024-01-21T22:23:17.613 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 190/193 2024-01-21T22:23:17.663 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 191/193 2024-01-21T22:23:17.731 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 192/193 2024-01-21T22:23:17.854 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 193/193 2024-01-21T22:23:18.111 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 193/193 2024-01-21T22:23:18.111 INFO:teuthology.orchestra.run.smithi138.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-01-21T22:23:18.111 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:27.579 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:19.0.0-811.geb1b049f.el9.x86_64 1/193 2024-01-21T22:23:27.579 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:19.0.0-811.geb1b049f.el9.x86_64 2/193 2024-01-21T22:23:27.580 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-debuginfo-2:19.0.0-811.geb1b049f.el9.x 3/193 2024-01-21T22:23:27.580 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:19.0.0-811.geb1b049f.el9.x86_64 4/193 2024-01-21T22:23:27.580 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-debuginfo-2:19.0.0-811.geb1b049f.el9 5/193 2024-01-21T22:23:27.580 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 6/193 2024-01-21T22:23:27.580 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:19.0.0-811.geb1b049f.el9.x86_64 7/193 2024-01-21T22:23:27.580 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-debuginfo-2:19.0.0-811.geb1b049f.el9.x 8/193 2024-01-21T22:23:27.580 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-811.geb1b04 9/193 2024-01-21T22:23:27.580 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:19.0.0-8 10/193 2024-01-21T22:23:27.580 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:19.0.0-811.geb1b049f.el9.x86_64 11/193 2024-01-21T22:23:27.580 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-debuginfo-2:19.0.0-811.geb1b049f.el9.x8 12/193 2024-01-21T22:23:27.580 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:19.0.0-811.geb1b049f.el9.x86_64 13/193 2024-01-21T22:23:27.581 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-debuginfo-2:19.0.0-811.geb1b049f.el9.x8 14/193 2024-01-21T22:23:27.581 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:19.0.0-811.geb1b049f.el9.x86_64 15/193 2024-01-21T22:23:27.581 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-debuginfo-2:19.0.0-811.geb1b049f.el9.x8 16/193 2024-01-21T22:23:27.581 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:19.0.0-811.geb1b049f.el9.x86_64 17/193 2024-01-21T22:23:27.581 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-debuginfo-2:19.0.0-811.geb1b049f.el9.x8 18/193 2024-01-21T22:23:27.581 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:19.0.0-811.geb1b049f.el9.x86_64 19/193 2024-01-21T22:23:27.581 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-debuginfo-2:19.0.0-811.geb1b049f.el 20/193 2024-01-21T22:23:27.581 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:19.0.0-811.geb1b049f.el9.x86_64 21/193 2024-01-21T22:23:27.581 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:19.0.0-811.geb1b049f.el9.x86_64 22/193 2024-01-21T22:23:27.581 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-debuginfo-2:19.0.0-811.geb1b049f.el9.x 23/193 2024-01-21T22:23:27.581 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-mirror-2:19.0.0-811.geb1b049f.el9.x86_64 24/193 2024-01-21T22:23:27.581 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-shell-2:19.0.0-811.geb1b049f.el9.x86_64 25/193 2024-01-21T22:23:27.581 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:19.0.0-811.geb1b049f.el9.x86_6 26/193 2024-01-21T22:23:27.581 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:19.0.0-811.geb1b049f.el9.x86_64 27/193 2024-01-21T22:23:27.582 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-debuginfo-2:19.0.0-811.geb1b049f.el9. 28/193 2024-01-21T22:23:27.582 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephsqlite-2:19.0.0-811.geb1b049f.el9.x86_64 29/193 2024-01-21T22:23:27.582 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:19.0.0-811.geb1b049f.el9.x86_64 30/193 2024-01-21T22:23:27.582 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-debuginfo-2:19.0.0-811.geb1b049f. 31/193 2024-01-21T22:23:27.582 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-debuginfo-2:19.0.0-811.geb1b049f.el9.x 32/193 2024-01-21T22:23:27.582 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:19.0.0-811.geb1b049f.el9.x86_ 33/193 2024-01-21T22:23:27.582 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-debuginfo-2:19.0.0-811.geb1b049f.el9.x86 34/193 2024-01-21T22:23:27.582 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:19.0.0-811.geb1b049f.el9.x86_64 35/193 2024-01-21T22:23:27.582 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:19.0.0-811.geb1b049f.el9 36/193 2024-01-21T22:23:27.582 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:19.0.0-811.geb1b049f.el9.x 37/193 2024-01-21T22:23:27.582 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:19.0.0-811.geb1b049f.el9.x86_64 38/193 2024-01-21T22:23:27.582 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-debuginfo-2:19.0.0-811.geb1b049f. 39/193 2024-01-21T22:23:27.582 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:19.0.0-811.geb1b049f.el9.x86_64 40/193 2024-01-21T22:23:27.583 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-debuginfo-2:19.0.0-811.geb1b049f.e 41/193 2024-01-21T22:23:27.583 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:19.0.0-811.geb1b049f.el9.x86_64 42/193 2024-01-21T22:23:27.583 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-debuginfo-2:19.0.0-811.geb1b049f.el9 43/193 2024-01-21T22:23:27.583 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:19.0.0-811.geb1b049f.el9.x86_64 44/193 2024-01-21T22:23:27.583 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-debuginfo-2:19.0.0-811.geb1b049f.el9 45/193 2024-01-21T22:23:27.583 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:19.0.0-811.geb1b049f.el9.x86_64 46/193 2024-01-21T22:23:27.583 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-debuginfo-2:19.0.0-811.geb1b049f.el9.x8 47/193 2024-01-21T22:23:27.583 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:19.0.0-811.geb1b049f.el9.x86_64 48/193 2024-01-21T22:23:27.583 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-debuginfo-2:19.0.0-811.geb1b049f.el9. 49/193 2024-01-21T22:23:27.583 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:19.0.0-811.geb1b049f.el9.x86_64 50/193 2024-01-21T22:23:27.583 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-debuginfo-2:19.0.0-811.geb1b049f.el9.x86 51/193 2024-01-21T22:23:27.583 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-811.geb1b049f.e 52/193 2024-01-21T22:23:27.583 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-811.geb1b049f.el9.noar 53/193 2024-01-21T22:23:27.583 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-811.geb1b049f.el9.no 54/193 2024-01-21T22:23:27.583 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-811.geb1b 55/193 2024-01-21T22:23:27.584 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-811.geb1b049f.el9 56/193 2024-01-21T22:23:27.584 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-rook-2:19.0.0-811.geb1b049f.el9.noarch 57/193 2024-01-21T22:23:27.584 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-811.geb1b049f.el 58/193 2024-01-21T22:23:27.584 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-volume-2:19.0.0-811.geb1b049f.el9.noarch 59/193 2024-01-21T22:23:27.584 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:19.0.0-811.geb1b049f.el9.noarch 60/193 2024-01-21T22:23:27.584 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-top-2:19.0.0-811.geb1b049f.el9.noarch 61/193 2024-01-21T22:23:27.584 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : automake-1.16.2-8.el9.noarch 62/193 2024-01-21T22:23:27.584 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : bison-3.7.4-5.el9.x86_64 63/193 2024-01-21T22:23:27.584 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 64/193 2024-01-21T22:23:27.584 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 65/193 2024-01-21T22:23:27.584 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flex-2.6.4-9.el9.x86_64 66/193 2024-01-21T22:23:27.584 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 67/193 2024-01-21T22:23:27.584 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 68/193 2024-01-21T22:23:27.585 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 69/193 2024-01-21T22:23:27.585 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 70/193 2024-01-21T22:23:27.585 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 71/193 2024-01-21T22:23:27.585 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 72/193 2024-01-21T22:23:27.585 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 73/193 2024-01-21T22:23:27.585 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 74/193 2024-01-21T22:23:27.585 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 75/193 2024-01-21T22:23:27.585 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 76/193 2024-01-21T22:23:27.585 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 77/193 2024-01-21T22:23:27.585 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-5.4.4-4.el9.x86_64 78/193 2024-01-21T22:23:27.585 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 79/193 2024-01-21T22:23:27.585 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 80/193 2024-01-21T22:23:27.585 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 81/193 2024-01-21T22:23:27.586 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-13.11-1.el9.x86_64 82/193 2024-01-21T22:23:27.586 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 83/193 2024-01-21T22:23:27.586 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 84/193 2024-01-21T22:23:27.586 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 85/193 2024-01-21T22:23:27.586 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 86/193 2024-01-21T22:23:27.586 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 87/193 2024-01-21T22:23:27.586 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 88/193 2024-01-21T22:23:27.586 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 89/193 2024-01-21T22:23:27.586 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 90/193 2024-01-21T22:23:27.586 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 91/193 2024-01-21T22:23:27.586 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 92/193 2024-01-21T22:23:27.586 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 93/193 2024-01-21T22:23:27.586 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/193 2024-01-21T22:23:27.586 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 95/193 2024-01-21T22:23:27.587 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 96/193 2024-01-21T22:23:27.587 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 97/193 2024-01-21T22:23:27.587 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 98/193 2024-01-21T22:23:27.587 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 99/193 2024-01-21T22:23:27.587 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 100/193 2024-01-21T22:23:27.587 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 101/193 2024-01-21T22:23:27.587 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 102/193 2024-01-21T22:23:27.587 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 103/193 2024-01-21T22:23:27.587 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 104/193 2024-01-21T22:23:27.587 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 105/193 2024-01-21T22:23:27.587 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 106/193 2024-01-21T22:23:27.587 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 107/193 2024-01-21T22:23:27.588 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 108/193 2024-01-21T22:23:27.588 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 109/193 2024-01-21T22:23:27.588 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : jq-1.6-16.el9.x86_64 110/193 2024-01-21T22:23:27.588 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 111/193 2024-01-21T22:23:27.588 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 112/193 2024-01-21T22:23:27.588 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 113/193 2024-01-21T22:23:27.588 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 114/193 2024-01-21T22:23:27.588 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 115/193 2024-01-21T22:23:27.588 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ply-3.11-14.el9.noarch 116/193 2024-01-21T22:23:27.588 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 117/193 2024-01-21T22:23:27.588 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 118/193 2024-01-21T22:23:27.588 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 119/193 2024-01-21T22:23:27.588 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 120/193 2024-01-21T22:23:27.589 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 121/193 2024-01-21T22:23:27.589 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 122/193 2024-01-21T22:23:27.589 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 123/193 2024-01-21T22:23:27.589 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 124/193 2024-01-21T22:23:27.589 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 125/193 2024-01-21T22:23:27.589 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 126/193 2024-01-21T22:23:27.589 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 127/193 2024-01-21T22:23:27.589 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 128/193 2024-01-21T22:23:27.589 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 129/193 2024-01-21T22:23:27.589 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 130/193 2024-01-21T22:23:27.589 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 131/193 2024-01-21T22:23:27.589 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 132/193 2024-01-21T22:23:27.589 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 133/193 2024-01-21T22:23:27.590 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 134/193 2024-01-21T22:23:27.590 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 135/193 2024-01-21T22:23:27.590 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 136/193 2024-01-21T22:23:27.590 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-google-auth-1:2.26.2-1.el9.noarch 137/193 2024-01-21T22:23:27.590 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 138/193 2024-01-21T22:23:27.590 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 139/193 2024-01-21T22:23:27.590 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 140/193 2024-01-21T22:23:27.590 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 141/193 2024-01-21T22:23:27.590 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 142/193 2024-01-21T22:23:27.590 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 143/193 2024-01-21T22:23:27.590 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 144/193 2024-01-21T22:23:27.590 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 145/193 2024-01-21T22:23:27.590 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 146/193 2024-01-21T22:23:27.590 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 147/193 2024-01-21T22:23:27.590 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 148/193 2024-01-21T22:23:27.591 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 149/193 2024-01-21T22:23:27.591 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 150/193 2024-01-21T22:23:27.591 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 151/193 2024-01-21T22:23:27.591 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 152/193 2024-01-21T22:23:27.591 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 153/193 2024-01-21T22:23:27.591 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 154/193 2024-01-21T22:23:27.591 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 155/193 2024-01-21T22:23:27.591 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 156/193 2024-01-21T22:23:27.591 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 157/193 2024-01-21T22:23:27.591 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 158/193 2024-01-21T22:23:27.591 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 159/193 2024-01-21T22:23:27.591 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : re2-1:20211101-3.el9.x86_64 160/193 2024-01-21T22:23:27.591 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 161/193 2024-01-21T22:23:27.592 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:19.0.0-811.geb1b049f.el9.x86_64 162/193 2024-01-21T22:23:27.592 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 163/193 2024-01-21T22:23:27.592 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:19.0.0-811.geb1b049f.el9.x86_64 164/193 2024-01-21T22:23:27.592 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 165/193 2024-01-21T22:23:27.592 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 166/193 2024-01-21T22:23:27.592 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 167/193 2024-01-21T22:23:27.592 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-1:1.45.10-1.el9.x86_64 168/193 2024-01-21T22:23:27.592 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 169/193 2024-01-21T22:23:27.592 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-libnm-1:1.45.10-1.el9.x86_64 170/193 2024-01-21T22:23:27.592 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 171/193 2024-01-21T22:23:27.592 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-team-1:1.45.10-1.el9.x86_64 172/193 2024-01-21T22:23:27.592 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 173/193 2024-01-21T22:23:27.593 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-tui-1:1.45.10-1.el9.x86_64 174/193 2024-01-21T22:23:27.593 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/193 2024-01-21T22:23:27.593 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : acl-2.3.1-4.el9.x86_64 176/193 2024-01-21T22:23:27.593 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : acl-2.3.1-3.el9.x86_64 177/193 2024-01-21T22:23:27.593 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 178/193 2024-01-21T22:23:27.593 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 179/193 2024-01-21T22:23:27.593 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 180/193 2024-01-21T22:23:27.593 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 181/193 2024-01-21T22:23:27.593 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 182/193 2024-01-21T22:23:27.593 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 183/193 2024-01-21T22:23:27.593 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 184/193 2024-01-21T22:23:27.593 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 185/193 2024-01-21T22:23:27.594 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 186/193 2024-01-21T22:23:27.594 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 187/193 2024-01-21T22:23:27.594 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 188/193 2024-01-21T22:23:27.594 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 189/193 2024-01-21T22:23:27.594 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 190/193 2024-01-21T22:23:27.594 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 191/193 2024-01-21T22:23:27.594 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 192/193 2024-01-21T22:23:30.197 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 193/193 2024-01-21T22:23:30.197 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:30.197 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2024-01-21T22:23:30.198 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-1:1.45.10-1.el9.x86_64 2024-01-21T22:23:30.198 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-libnm-1:1.45.10-1.el9.x86_64 2024-01-21T22:23:30.198 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-team-1:1.45.10-1.el9.x86_64 2024-01-21T22:23:30.198 INFO:teuthology.orchestra.run.smithi138.stdout: NetworkManager-tui-1:1.45.10-1.el9.x86_64 2024-01-21T22:23:30.198 INFO:teuthology.orchestra.run.smithi138.stdout: acl-2.3.1-4.el9.x86_64 2024-01-21T22:23:30.198 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-01-21T22:23:30.198 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-01-21T22:23:30.198 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-01-21T22:23:30.198 INFO:teuthology.orchestra.run.smithi138.stdout: libacl-2.3.1-4.el9.x86_64 2024-01-21T22:23:30.198 INFO:teuthology.orchestra.run.smithi138.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-01-21T22:23:30.198 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.198 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.198 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-1:3.0.7-25.el9.x86_64 2024-01-21T22:23:30.198 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2024-01-21T22:23:30.199 INFO:teuthology.orchestra.run.smithi138.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-01-21T22:23:30.199 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-01-21T22:23:30.199 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-01-21T22:23:30.199 INFO:teuthology.orchestra.run.smithi138.stdout: automake-1.16.2-8.el9.noarch 2024-01-21T22:23:30.199 INFO:teuthology.orchestra.run.smithi138.stdout: bison-3.7.4-5.el9.x86_64 2024-01-21T22:23:30.199 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-01-21T22:23:30.199 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.199 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.199 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.199 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.199 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.199 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.199 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.199 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.199 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards-2:19.0.0-811.geb1b049f.el9.noarch 2024-01-21T22:23:30.199 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.200 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.200 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.200 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.200 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.200 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm-2:19.0.0-811.geb1b049f.el9.noarch 2024-01-21T22:23:30.200 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard-2:19.0.0-811.geb1b049f.el9.noarch 2024-01-21T22:23:30.200 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.200 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local-2:19.0.0-811.geb1b049f.el9.noarch 2024-01-21T22:23:30.200 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:19.0.0-811.geb1b049f.el9.noarch 2024-01-21T22:23:30.200 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook-2:19.0.0-811.geb1b049f.el9.noarch 2024-01-21T22:23:30.200 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.200 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.200 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.201 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.201 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts-2:19.0.0-811.geb1b049f.el9.noarch 2024-01-21T22:23:30.201 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.201 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.201 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.201 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.201 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.201 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume-2:19.0.0-811.geb1b049f.el9.noarch 2024-01-21T22:23:30.201 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:19.0.0-811.geb1b049f.el9.noarch 2024-01-21T22:23:30.201 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.201 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.201 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top-2:19.0.0-811.geb1b049f.el9.noarch 2024-01-21T22:23:30.201 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-01-21T22:23:30.201 INFO:teuthology.orchestra.run.smithi138.stdout: flex-2.6.4-9.el9.x86_64 2024-01-21T22:23:30.202 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-01-21T22:23:30.202 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-01-21T22:23:30.202 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-01-21T22:23:30.202 INFO:teuthology.orchestra.run.smithi138.stdout: fuse-2.9.9-15.el9.x86_64 2024-01-21T22:23:30.202 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-01-21T22:23:30.202 INFO:teuthology.orchestra.run.smithi138.stdout: jq-1.6-16.el9.x86_64 2024-01-21T22:23:30.202 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-01-21T22:23:30.202 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-9.0.0-10.el9.x86_64 2024-01-21T22:23:30.202 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-01-21T22:23:30.202 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.202 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.202 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.202 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.202 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-01-21T22:23:30.203 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.7-2.el9.x86_64 2024-01-21T22:23:30.203 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-01-21T22:23:30.203 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-01-21T22:23:30.203 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-01-21T22:23:30.203 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.203 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.203 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.203 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.203 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.203 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-01-21T22:23:30.203 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.203 INFO:teuthology.orchestra.run.smithi138.stdout: libtool-2.4.6-45.el9.x86_64 2024-01-21T22:23:30.203 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.6.2-1.el9.x86_64 2024-01-21T22:23:30.203 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2024-01-21T22:23:30.203 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-01-21T22:23:30.203 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-01-21T22:23:30.204 INFO:teuthology.orchestra.run.smithi138.stdout: lua-5.4.4-4.el9.x86_64 2024-01-21T22:23:30.204 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-01-21T22:23:30.204 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks-3.9.2-1.el9.noarch 2024-01-21T22:23:30.204 INFO:teuthology.orchestra.run.smithi138.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-01-21T22:23:30.204 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-0.3.21-2.el9.x86_64 2024-01-21T22:23:30.204 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-01-21T22:23:30.204 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2024-01-21T22:23:30.204 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-01-21T22:23:30.204 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-13.11-1.el9.x86_64 2024-01-21T22:23:30.204 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-01-21T22:23:30.204 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-01-21T22:23:30.204 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server-13.11-1.el9.x86_64 2024-01-21T22:23:30.204 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-01-21T22:23:30.204 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-01-21T22:23:30.205 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel-2.9.1-2.el9.noarch 2024-01-21T22:23:30.205 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-01-21T22:23:30.205 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-01-21T22:23:30.205 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.205 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.205 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.205 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.205 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-01-21T22:23:30.205 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-01-21T22:23:30.205 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-01-21T22:23:30.205 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-01-21T22:23:30.205 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-01-21T22:23:30.205 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-01-21T22:23:30.205 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-01-21T22:23:30.205 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-01-21T22:23:30.206 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth-1:2.26.2-1.el9.noarch 2024-01-21T22:23:30.206 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-01-21T22:23:30.206 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-01-21T22:23:30.206 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-01-21T22:23:30.206 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-01-21T22:23:30.206 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-01-21T22:23:30.206 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-01-21T22:23:30.206 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-01-21T22:23:30.206 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-01-21T22:23:30.206 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-01-21T22:23:30.206 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-01-21T22:23:30.206 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.1.4-6.el9.noarch 2024-01-21T22:23:30.206 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-01-21T22:23:30.206 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-01-21T22:23:30.206 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-01-21T22:23:30.206 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-01-21T22:23:30.207 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-01-21T22:23:30.207 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-01-21T22:23:30.207 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-01-21T22:23:30.207 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply-3.11-14.el9.noarch 2024-01-21T22:23:30.207 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-3.1.0-2.el9.noarch 2024-01-21T22:23:30.207 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-01-21T22:23:30.207 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-01-21T22:23:30.207 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-01-21T22:23:30.207 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-01-21T22:23:30.207 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser-2.20-6.el9.noarch 2024-01-21T22:23:30.207 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-01-21T22:23:30.207 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2021.1-5.el9.noarch 2024-01-21T22:23:30.207 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.207 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.207 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.207 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.208 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-01-21T22:23:30.208 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-2.25.1-7.el9.noarch 2024-01-21T22:23:30.208 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-01-21T22:23:30.208 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.208 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.208 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes-2.5.1-5.el9.noarch 2024-01-21T22:23:30.208 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa-4.9-2.el9.noarch 2024-01-21T22:23:30.208 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-01-21T22:23:30.208 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-01-21T22:23:30.208 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml-0.10.2-6.el9.noarch 2024-01-21T22:23:30.208 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-01-21T22:23:30.208 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-01-21T22:23:30.208 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-01-21T22:23:30.208 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.7-6.el9.noarch 2024-01-21T22:23:30.208 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-01-21T22:23:30.209 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-01-21T22:23:30.209 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-01-21T22:23:30.209 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.209 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.209 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.209 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.209 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.209 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-debuginfo-2:19.0.0-811.geb1b049f.el9.x86_64 2024-01-21T22:23:30.209 INFO:teuthology.orchestra.run.smithi138.stdout: re2-1:20211101-3.el9.x86_64 2024-01-21T22:23:30.209 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.4.1-5.el9.x86_64 2024-01-21T22:23:30.209 INFO:teuthology.orchestra.run.smithi138.stdout: thrift-0.15.0-2.el9.x86_64 2024-01-21T22:23:30.209 INFO:teuthology.orchestra.run.smithi138.stdout: uuid-1.6.2-55.el9.x86_64 2024-01-21T22:23:30.209 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-01-21T22:23:30.209 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-01-21T22:23:30.210 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-01-21T22:23:30.210 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:30.210 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-01-21T22:23:31.641 DEBUG:teuthology.parallel:result is None 2024-01-21T22:23:31.641 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=eb1b049f9003ac37e0a3ab33496ad1924d301254 2024-01-21T22:23:31.788 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-21T22:23:31.857 INFO:teuthology.orchestra.run.smithi138.stdout:19.0.0-811.geb1b049f.el9 2024-01-21T22:23:31.858 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-811.geb1b049f.el9 2024-01-21T22:23:31.858 INFO:teuthology.task.install:The correct ceph version 19.0.0-811.geb1b049f is installed. 2024-01-21T22:23:31.860 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=eb1b049f9003ac37e0a3ab33496ad1924d301254 2024-01-21T22:23:32.084 DEBUG:teuthology.orchestra.run.smithi152:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-01-21T22:23:32.110 INFO:teuthology.orchestra.run.smithi152.stdout:19.0.0-811.geb1b049f.el9 2024-01-21T22:23:32.110 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-811.geb1b049f.el9 2024-01-21T22:23:32.110 INFO:teuthology.task.install:The correct ceph version 19.0.0-811.geb1b049f is installed. 2024-01-21T22:23:32.112 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-01-21T22:23:32.112 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:23:32.113 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-21T22:23:32.214 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-21T22:23:32.214 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-01-21T22:23:32.250 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-01-21T22:23:32.250 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:23:32.250 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2024-01-21T22:23:32.341 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-21T22:23:32.480 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-21T22:23:32.480 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/daemon-helper 2024-01-21T22:23:32.513 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-01-21T22:23:32.583 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-01-21T22:23:32.583 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:23:32.583 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-21T22:23:32.675 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-21T22:23:32.813 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-21T22:23:32.814 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/adjust-ulimits 2024-01-21T22:23:32.841 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-01-21T22:23:32.909 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-01-21T22:23:32.909 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:23:32.909 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2024-01-21T22:23:33.000 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-21T22:23:33.086 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-21T22:23:33.086 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/usr/bin/stdin-killer 2024-01-21T22:23:33.113 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-01-21T22:23:33.180 INFO:teuthology.run_tasks:Running task ceph... 2024-01-21T22:23:33.263 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-01-21T22:23:33.263 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /var/log/ceph 2024-01-21T22:23:33.270 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 777 /var/log/ceph 2024-01-21T22:23:33.360 INFO:tasks.ceph:Disabling ceph logrotate... 2024-01-21T22:23:33.360 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-21T22:23:33.367 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /etc/logrotate.d/ceph 2024-01-21T22:23:33.451 INFO:tasks.ceph:Creating extra log directories... 2024-01-21T22:23:33.452 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-21T22:23:33.459 DEBUG:teuthology.orchestra.run.smithi152:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-01-21T22:23:33.548 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-01-21T22:23:33.548 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-01-21T22:23:33.548 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-01-21T22:23:33.549 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:23:33.549 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-01-21T22:23:33.603 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-01-21T22:23:33.867 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-01-21T22:23:33.962 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-21T22:23:33.962 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-01-21T22:23:33.977 DEBUG:teuthology.orchestra.run.smithi152:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-01-21T22:23:34.096 DEBUG:teuthology.orchestra.run.smithi152:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-01-21T22:23:34.128 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-01-21T22:23:34.128 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mds beacon grace': 60, 'mds heartbeat grace': 60, 'ms die on skipped message': False, 'osd heartbeat grace': 60}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds valgrind exit': True, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd crush smoke test': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow request', 'SLOW_OPS', 'MON_DOWN'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-01-21T22:23:34.129 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2024-01-19_10:13:34-fs-wip-vshankar-testing-20240119.075157-testing-default-smithi/7522024', 'branch': 'wip-vshankar-testing-20240119.075157', 'description': 'fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/dbench validater/valgrind}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7522024', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '06652275eb934672217aaf9d4180f4a90395eb16'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2024-01-19_10:13:34-fs-wip-vshankar-testing-20240119.075157-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-20240119.075157'}, 'ceph': {'cephfs': {'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mds beacon grace': 60, 'mds heartbeat grace': 60, 'ms die on skipped message': False, 'osd heartbeat grace': 60}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds valgrind exit': True, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd crush smoke test': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow request', 'SLOW_OPS', 'MON_DOWN'], 'sha1': 'eb1b049f9003ac37e0a3ab33496ad1924d301254', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes']}, 'watchdog': {'daemon_restart': 'normal'}}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'ceph-fuse': {'client.0': {'valgrind': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes']}}, 'install': {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': 'eb1b049f9003ac37e0a3ab33496ad1924d301254'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-20240119.075157', 'sha1': 'eb1b049f9003ac37e0a3ab33496ad1924d301254'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 90, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 3392, 'sha1': 'eb1b049f9003ac37e0a3ab33496ad1924d301254', 'sleep_before_teardown': 0, 'subset': '163/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-20240119.075157', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_eb1b049f9003ac37e0a3ab33496ad1924d301254/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'eb1b049f9003ac37e0a3ab33496ad1924d301254', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi138.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLyJGSaW2X8YY71fG8SngBVERcLbGclRlIadn2c0LipJRJCrI6qJ6FL8dD81NbSnvf5uuIDaFsJZ2adGvW7BFE4=', 'smithi152.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEeNVGhw4BYJUYb1i7A07lKn6tOaA9Y9fy0bgq+8084xBMhXWcHRfdSuRm+R4HCb2qW/u8uwFcdu2WAnl5J4ySI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '06652275eb934672217aaf9d4180f4a90395eb16'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'debuginfo': True, 'flavor': 'default', 'sha1': 'eb1b049f9003ac37e0a3ab33496ad1924d301254'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mds beacon grace': 60, 'mds heartbeat grace': 60, 'ms die on skipped message': False, 'osd heartbeat grace': 60}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds valgrind exit': True, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd crush smoke test': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow request', 'SLOW_OPS', 'MON_DOWN'], 'sha1': 'eb1b049f9003ac37e0a3ab33496ad1924d301254', 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes']}, 'watchdog': {'daemon_restart': 'normal'}, 'cluster': 'ceph'}}, {'kclient': None}, {'workunit': {'clients': {'all': ['suites/dbench.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': 'cd45576300487d997e5a85abed65500b9f5d143b', 'timestamp': '2024-01-19_10:13:34', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-01-21T22:23:34.129 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-21T22:23:34.137 DEBUG:teuthology.orchestra.run.smithi152:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-01-21T22:23:34.186 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m0777 -- /var/run/ceph 2024-01-21T22:23:34.191 DEBUG:teuthology.orchestra.run.smithi152:> sudo install -d -m0777 -- /var/run/ceph 2024-01-21T22:23:34.272 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:23:34.272 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-01-21T22:23:34.325 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-21T22:23:34.325 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-01-21T22:23:34.419 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-21T22:23:34.420 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-21T22:23:34.420 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 632 Links: 1 2024-01-21T22:23:34.420 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-21T22:23:34.420 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-01-21T22:23:34.420 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-01-21 22:23:18.567285244 +0000 2024-01-21T22:23:34.420 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-01-21 22:14:28.276929775 +0000 2024-01-21T22:23:34.420 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-01-21 22:14:28.276929775 +0000 2024-01-21T22:23:34.420 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-01-21 22:14:28.276929775 +0000 2024-01-21T22:23:34.421 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-21T22:23:34.508 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-01-21T22:23:34.508 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-01-21T22:23:34.508 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000327084 s, 1.6 MB/s 2024-01-21T22:23:34.512 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-21T22:23:34.568 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-01-21T22:23:34.621 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-21T22:23:34.621 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-21T22:23:34.621 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 650 Links: 1 2024-01-21T22:23:34.622 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-21T22:23:34.622 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-01-21T22:23:34.622 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-01-21 22:23:18.567285244 +0000 2024-01-21T22:23:34.622 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-01-21 22:14:28.861917374 +0000 2024-01-21T22:23:34.622 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-01-21 22:14:28.861917374 +0000 2024-01-21T22:23:34.622 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-01-21 22:14:28.861917374 +0000 2024-01-21T22:23:34.622 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-21T22:23:34.712 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-01-21T22:23:34.712 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-01-21T22:23:34.712 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000341275 s, 1.5 MB/s 2024-01-21T22:23:34.716 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-21T22:23:34.771 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-01-21T22:23:34.822 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-21T22:23:34.822 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-21T22:23:34.822 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 672 Links: 1 2024-01-21T22:23:34.822 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-21T22:23:34.822 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-01-21T22:23:34.822 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-01-21 22:23:18.568285223 +0000 2024-01-21T22:23:34.823 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-01-21 22:14:29.394906075 +0000 2024-01-21T22:23:34.823 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-01-21 22:14:29.394906075 +0000 2024-01-21T22:23:34.823 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-01-21 22:14:29.394906075 +0000 2024-01-21T22:23:34.823 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-21T22:23:34.904 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-01-21T22:23:34.904 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-01-21T22:23:34.904 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000335198 s, 1.5 MB/s 2024-01-21T22:23:34.908 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-21T22:23:34.962 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-01-21T22:23:35.011 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-21T22:23:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-21T22:23:35.012 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 689 Links: 1 2024-01-21T22:23:35.012 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-21T22:23:35.012 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-01-21T22:23:35.012 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-01-21 22:23:18.569285202 +0000 2024-01-21T22:23:35.012 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-01-21 22:14:29.930894713 +0000 2024-01-21T22:23:35.012 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-01-21 22:14:29.930894713 +0000 2024-01-21T22:23:35.012 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-01-21 22:14:29.930894713 +0000 2024-01-21T22:23:35.012 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-21T22:23:35.097 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-01-21T22:23:35.097 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-01-21T22:23:35.097 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000337665 s, 1.5 MB/s 2024-01-21T22:23:35.101 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-21T22:23:35.157 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-21T22:23:35.157 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-21T22:23:35.157 DEBUG:teuthology.orchestra.run.smithi152:> dd if=/scratch_devs of=/dev/stdout 2024-01-21T22:23:35.175 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-01-21T22:23:35.175 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_1 2024-01-21T22:23:35.232 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-01-21T22:23:35.232 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-21T22:23:35.232 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-01-21T22:23:35.232 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-21T22:23:35.232 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-01-21T22:23:35.232 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-01-21 22:22:57.959446006 +0000 2024-01-21T22:23:35.232 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-01-21 22:13:20.436079295 +0000 2024-01-21T22:23:35.232 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-01-21 22:13:20.436079295 +0000 2024-01-21T22:23:35.233 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-01-21 22:13:20.436079295 +0000 2024-01-21T22:23:35.233 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-01-21T22:23:35.300 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-01-21T22:23:35.300 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-01-21T22:23:35.300 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000307438 s, 1.7 MB/s 2024-01-21T22:23:35.302 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-01-21T22:23:35.359 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_2 2024-01-21T22:23:35.416 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-01-21T22:23:35.417 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-21T22:23:35.417 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-01-21T22:23:35.417 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-21T22:23:35.417 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-01-21T22:23:35.417 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-01-21 22:22:57.959446006 +0000 2024-01-21T22:23:35.417 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-01-21 22:13:20.940073312 +0000 2024-01-21T22:23:35.417 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-01-21 22:13:20.940073312 +0000 2024-01-21T22:23:35.417 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-01-21 22:13:20.940073312 +0000 2024-01-21T22:23:35.417 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-01-21T22:23:35.484 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-01-21T22:23:35.484 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-01-21T22:23:35.485 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000195789 s, 2.6 MB/s 2024-01-21T22:23:35.486 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-01-21T22:23:35.542 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_3 2024-01-21T22:23:35.597 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-01-21T22:23:35.597 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-21T22:23:35.597 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-01-21T22:23:35.597 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-21T22:23:35.598 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-01-21T22:23:35.598 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-01-21 22:22:57.960445995 +0000 2024-01-21T22:23:35.598 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-01-21 22:13:21.279069289 +0000 2024-01-21T22:23:35.598 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-01-21 22:13:21.279069289 +0000 2024-01-21T22:23:35.598 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-01-21 22:13:21.279069289 +0000 2024-01-21T22:23:35.598 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-01-21T22:23:35.663 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-01-21T22:23:35.663 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-01-21T22:23:35.663 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000163961 s, 3.1 MB/s 2024-01-21T22:23:35.665 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-01-21T22:23:35.721 DEBUG:teuthology.orchestra.run.smithi152:> stat /dev/vg_nvme/lv_4 2024-01-21T22:23:35.775 INFO:teuthology.orchestra.run.smithi152.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-01-21T22:23:35.776 INFO:teuthology.orchestra.run.smithi152.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-01-21T22:23:35.776 INFO:teuthology.orchestra.run.smithi152.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-01-21T22:23:35.776 INFO:teuthology.orchestra.run.smithi152.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-21T22:23:35.776 INFO:teuthology.orchestra.run.smithi152.stdout:Context: system_u:object_r:device_t:s0 2024-01-21T22:23:35.776 INFO:teuthology.orchestra.run.smithi152.stdout:Access: 2024-01-21 22:22:57.960445995 +0000 2024-01-21T22:23:35.776 INFO:teuthology.orchestra.run.smithi152.stdout:Modify: 2024-01-21 22:13:21.617065282 +0000 2024-01-21T22:23:35.776 INFO:teuthology.orchestra.run.smithi152.stdout:Change: 2024-01-21 22:13:21.617065282 +0000 2024-01-21T22:23:35.776 INFO:teuthology.orchestra.run.smithi152.stdout: Birth: 2024-01-21 22:13:21.617065282 +0000 2024-01-21T22:23:35.776 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-01-21T22:23:35.841 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records in 2024-01-21T22:23:35.841 INFO:teuthology.orchestra.run.smithi152.stderr:1+0 records out 2024-01-21T22:23:35.841 INFO:teuthology.orchestra.run.smithi152.stderr:512 bytes copied, 0.000203913 s, 2.5 MB/s 2024-01-21T22:23:35.842 DEBUG:teuthology.orchestra.run.smithi152:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-01-21T22:23:35.898 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-21T22:23:35.898 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi152.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-01-21T22:23:35.898 INFO:tasks.ceph:Generating config... 2024-01-21T22:23:35.900 INFO:tasks.ceph:[client] client mount timeout = 600 2024-01-21T22:23:35.900 INFO:tasks.ceph:[client] debug client = 20 2024-01-21T22:23:35.900 INFO:tasks.ceph:[client] debug ms = 1 2024-01-21T22:23:35.900 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-01-21T22:23:35.900 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-01-21T22:23:35.900 INFO:tasks.ceph:[global] mds beacon grace = 60 2024-01-21T22:23:35.900 INFO:tasks.ceph:[global] mds heartbeat grace = 60 2024-01-21T22:23:35.900 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-01-21T22:23:35.900 INFO:tasks.ceph:[global] osd heartbeat grace = 60 2024-01-21T22:23:35.900 INFO:tasks.ceph:[mds] debug mds = 20 2024-01-21T22:23:35.900 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-01-21T22:23:35.900 INFO:tasks.ceph:[mds] debug ms = 1 2024-01-21T22:23:35.900 INFO:tasks.ceph:[mds] mds debug frag = True 2024-01-21T22:23:35.901 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-01-21T22:23:35.901 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-01-21T22:23:35.901 INFO:tasks.ceph:[mds] mds valgrind exit = True 2024-01-21T22:23:35.901 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-01-21T22:23:35.901 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-01-21T22:23:35.901 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-01-21T22:23:35.901 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-01-21T22:23:35.901 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-01-21T22:23:35.901 INFO:tasks.ceph:[mgr] debug ms = 1 2024-01-21T22:23:35.901 INFO:tasks.ceph:[mon] debug mon = 20 2024-01-21T22:23:35.901 INFO:tasks.ceph:[mon] debug ms = 1 2024-01-21T22:23:35.901 INFO:tasks.ceph:[mon] debug paxos = 20 2024-01-21T22:23:35.901 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-01-21T22:23:35.901 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2024-01-21T22:23:35.901 INFO:tasks.ceph:[osd] bdev async discard = True 2024-01-21T22:23:35.901 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-01-21T22:23:35.902 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-01-21T22:23:35.902 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-01-21T22:23:35.902 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-01-21T22:23:35.902 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-01-21T22:23:35.902 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-01-21T22:23:35.902 INFO:tasks.ceph:[osd] debug ms = 1 2024-01-21T22:23:35.902 INFO:tasks.ceph:[osd] debug osd = 20 2024-01-21T22:23:35.902 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-01-21T22:23:35.902 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-01-21T22:23:35.902 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-01-21T22:23:35.902 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-01-21T22:23:35.902 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-01-21T22:23:35.902 INFO:tasks.ceph:[osd] osd fast shutdown = False 2024-01-21T22:23:35.902 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-01-21T22:23:35.902 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-01-21T22:23:35.903 INFO:tasks.ceph:Setting up mon.a... 2024-01-21T22:23:35.903 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-01-21T22:23:36.050 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.keyring 2024-01-21T22:23:36.058 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-01-21T22:23:36.190 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-21T22:23:36.283 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.138'), ('mon.b', '172.21.15.152'), ('mon.c', '[v2:172.21.15.152:3301,v1:172.21.15.152:6790]')] 2024-01-21T22:23:36.283 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.138,172.21.15.152,[v2:172.21.15.152:3301,v1:172.21.15.152:6790]', 'mds beacon grace': 60, 'mds heartbeat grace': 60, 'ms die on skipped message': False, 'osd heartbeat grace': 60}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd fast shutdown': False, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon osd crush smoke test': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds valgrind exit': True, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-01-21T22:23:36.284 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:23:36.284 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-01-21T22:23:36.333 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.138 --add b 172.21.15.152 --addv c '[v2:172.21.15.152:3301,v1:172.21.15.152:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-01-21T22:23:36.484 INFO:teuthology.orchestra.run.smithi138.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-01-21T22:23:36.484 INFO:teuthology.orchestra.run.smithi138.stdout:monmaptool: generated fsid d488d446-149f-48d0-89ca-b2830ae72846 2024-01-21T22:23:36.485 INFO:teuthology.orchestra.run.smithi138.stdout:setting min_mon_release = quincy 2024-01-21T22:23:36.485 INFO:teuthology.orchestra.run.smithi138.stdout:epoch 0 2024-01-21T22:23:36.485 INFO:teuthology.orchestra.run.smithi138.stdout:fsid d488d446-149f-48d0-89ca-b2830ae72846 2024-01-21T22:23:36.485 INFO:teuthology.orchestra.run.smithi138.stdout:last_changed 2024-01-21T22:23:36.483851+0000 2024-01-21T22:23:36.485 INFO:teuthology.orchestra.run.smithi138.stdout:created 2024-01-21T22:23:36.483851+0000 2024-01-21T22:23:36.485 INFO:teuthology.orchestra.run.smithi138.stdout:min_mon_release 17 (quincy) 2024-01-21T22:23:36.485 INFO:teuthology.orchestra.run.smithi138.stdout:election_strategy: 1 2024-01-21T22:23:36.485 INFO:teuthology.orchestra.run.smithi138.stdout:0: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.a 2024-01-21T22:23:36.485 INFO:teuthology.orchestra.run.smithi138.stdout:1: [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] mon.b 2024-01-21T22:23:36.485 INFO:teuthology.orchestra.run.smithi138.stdout:2: [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] mon.c 2024-01-21T22:23:36.485 INFO:teuthology.orchestra.run.smithi138.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-01-21T22:23:36.487 DEBUG:teuthology.orchestra.run.smithi138:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-01-21T22:23:36.537 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d488d446-149f-48d0-89ca-b2830ae72846... 2024-01-21T22:23:36.537 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-21T22:23:36.544 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-01-21T22:23:36.596 INFO:teuthology.orchestra.run.smithi152.stdout:[global] 2024-01-21T22:23:36.596 INFO:teuthology.orchestra.run.smithi152.stdout: chdir = "" 2024-01-21T22:23:36.596 INFO:teuthology.orchestra.run.smithi152.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-21T22:23:36.596 INFO:teuthology.orchestra.run.smithi152.stdout: auth supported = cephx 2024-01-21T22:23:36.596 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.596 INFO:teuthology.orchestra.run.smithi152.stdout: filestore xattr use omap = true 2024-01-21T22:23:36.596 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.596 INFO:teuthology.orchestra.run.smithi152.stdout: mon clock drift allowed = 1.000 2024-01-21T22:23:36.597 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.597 INFO:teuthology.orchestra.run.smithi152.stdout: osd crush chooseleaf type = 0 2024-01-21T22:23:36.597 INFO:teuthology.orchestra.run.smithi152.stdout: auth debug = true 2024-01-21T22:23:36.597 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.597 INFO:teuthology.orchestra.run.smithi152.stdout: ms die on old message = true 2024-01-21T22:23:36.597 INFO:teuthology.orchestra.run.smithi152.stdout: ms die on bug = true 2024-01-21T22:23:36.597 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.597 INFO:teuthology.orchestra.run.smithi152.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-21T22:23:36.597 INFO:teuthology.orchestra.run.smithi152.stdout: mon pg warn max object skew = 0 2024-01-21T22:23:36.597 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.597 INFO:teuthology.orchestra.run.smithi152.stdout: # disable pg_autoscaler by default for new pools 2024-01-21T22:23:36.597 INFO:teuthology.orchestra.run.smithi152.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-21T22:23:36.597 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.597 INFO:teuthology.orchestra.run.smithi152.stdout: osd pool default size = 2 2024-01-21T22:23:36.598 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.598 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd allow primary affinity = true 2024-01-21T22:23:36.598 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd allow pg remap = true 2024-01-21T22:23:36.598 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on legacy crush tunables = false 2024-01-21T22:23:36.598 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on crush straw calc version zero = false 2024-01-21T22:23:36.598 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on no sortbitwise = false 2024-01-21T22:23:36.598 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on osd down out interval zero = false 2024-01-21T22:23:36.598 INFO:teuthology.orchestra.run.smithi152.stdout: mon warn on too few osds = false 2024-01-21T22:23:36.598 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-21T22:23:36.598 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-21T22:23:36.598 INFO:teuthology.orchestra.run.smithi152.stdout: mon_allow_pool_size_one = true 2024-01-21T22:23:36.598 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.598 INFO:teuthology.orchestra.run.smithi152.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-21T22:23:36.598 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.599 INFO:teuthology.orchestra.run.smithi152.stdout: osd default data pool replay window = 5 2024-01-21T22:23:36.599 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.599 INFO:teuthology.orchestra.run.smithi152.stdout: mon allow pool delete = true 2024-01-21T22:23:36.599 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.599 INFO:teuthology.orchestra.run.smithi152.stdout: mon cluster log file level = debug 2024-01-21T22:23:36.599 INFO:teuthology.orchestra.run.smithi152.stdout: debug asserts on shutdown = true 2024-01-21T22:23:36.599 INFO:teuthology.orchestra.run.smithi152.stdout: mon health detail to clog = false 2024-01-21T22:23:36.599 INFO:teuthology.orchestra.run.smithi152.stdout: mon host = "172.21.15.138,172.21.15.152,[v2:172.21.15.152:3301,v1:172.21.15.152:6790]" 2024-01-21T22:23:36.599 INFO:teuthology.orchestra.run.smithi152.stdout: mds beacon grace = 60 2024-01-21T22:23:36.599 INFO:teuthology.orchestra.run.smithi152.stdout: mds heartbeat grace = 60 2024-01-21T22:23:36.599 INFO:teuthology.orchestra.run.smithi152.stdout: ms die on skipped message = False 2024-01-21T22:23:36.599 INFO:teuthology.orchestra.run.smithi152.stdout: osd heartbeat grace = 60 2024-01-21T22:23:36.599 INFO:teuthology.orchestra.run.smithi152.stdout: fsid = d488d446-149f-48d0-89ca-b2830ae72846 2024-01-21T22:23:36.599 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.599 INFO:teuthology.orchestra.run.smithi152.stdout:[osd] 2024-01-21T22:23:36.600 INFO:teuthology.orchestra.run.smithi152.stdout: osd journal size = 100 2024-01-21T22:23:36.600 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.600 INFO:teuthology.orchestra.run.smithi152.stdout: osd scrub load threshold = 5.0 2024-01-21T22:23:36.600 INFO:teuthology.orchestra.run.smithi152.stdout: osd scrub max interval = 600 2024-01-21T22:23:36.600 INFO:teuthology.orchestra.run.smithi152.stdout: osd mclock profile = high_recovery_ops 2024-01-21T22:23:36.600 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.600 INFO:teuthology.orchestra.run.smithi152.stdout: osd recover clone overlap = true 2024-01-21T22:23:36.600 INFO:teuthology.orchestra.run.smithi152.stdout: osd recovery max chunk = 1048576 2024-01-21T22:23:36.600 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.600 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug shutdown = true 2024-01-21T22:23:36.600 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug op order = true 2024-01-21T22:23:36.600 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug verify stray on activate = true 2024-01-21T22:23:36.600 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug trim objects = true 2024-01-21T22:23:36.600 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.601 INFO:teuthology.orchestra.run.smithi152.stdout: osd open classes on start = true 2024-01-21T22:23:36.601 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug pg log writeout = true 2024-01-21T22:23:36.601 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.601 INFO:teuthology.orchestra.run.smithi152.stdout: osd deep scrub update digest min age = 30 2024-01-21T22:23:36.601 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.601 INFO:teuthology.orchestra.run.smithi152.stdout: osd map max advance = 10 2024-01-21T22:23:36.601 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.601 INFO:teuthology.orchestra.run.smithi152.stdout: journal zero on create = true 2024-01-21T22:23:36.601 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.601 INFO:teuthology.orchestra.run.smithi152.stdout: filestore ondisk finisher threads = 3 2024-01-21T22:23:36.601 INFO:teuthology.orchestra.run.smithi152.stdout: filestore apply finisher threads = 3 2024-01-21T22:23:36.601 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.601 INFO:teuthology.orchestra.run.smithi152.stdout: bdev debug aio = true 2024-01-21T22:23:36.601 INFO:teuthology.orchestra.run.smithi152.stdout: osd debug misdirected ops = true 2024-01-21T22:23:36.601 INFO:teuthology.orchestra.run.smithi152.stdout: bdev async discard = True 2024-01-21T22:23:36.602 INFO:teuthology.orchestra.run.smithi152.stdout: bdev enable discard = True 2024-01-21T22:23:36.602 INFO:teuthology.orchestra.run.smithi152.stdout: bluestore allocator = bitmap 2024-01-21T22:23:36.602 INFO:teuthology.orchestra.run.smithi152.stdout: bluestore block size = 96636764160 2024-01-21T22:23:36.602 INFO:teuthology.orchestra.run.smithi152.stdout: bluestore fsck on mount = True 2024-01-21T22:23:36.602 INFO:teuthology.orchestra.run.smithi152.stdout: debug bluefs = 1/20 2024-01-21T22:23:36.602 INFO:teuthology.orchestra.run.smithi152.stdout: debug bluestore = 1/20 2024-01-21T22:23:36.602 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2024-01-21T22:23:36.602 INFO:teuthology.orchestra.run.smithi152.stdout: debug osd = 20 2024-01-21T22:23:36.602 INFO:teuthology.orchestra.run.smithi152.stdout: debug rocksdb = 4/10 2024-01-21T22:23:36.602 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd backfillfull_ratio = 0.85 2024-01-21T22:23:36.602 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd full ratio = 0.9 2024-01-21T22:23:36.602 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd nearfull ratio = 0.8 2024-01-21T22:23:36.602 INFO:teuthology.orchestra.run.smithi152.stdout: osd failsafe full ratio = 0.95 2024-01-21T22:23:36.602 INFO:teuthology.orchestra.run.smithi152.stdout: osd fast shutdown = False 2024-01-21T22:23:36.603 INFO:teuthology.orchestra.run.smithi152.stdout: osd objectstore = bluestore 2024-01-21T22:23:36.603 INFO:teuthology.orchestra.run.smithi152.stdout: osd op complaint time = 180 2024-01-21T22:23:36.603 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.603 INFO:teuthology.orchestra.run.smithi152.stdout:[mgr] 2024-01-21T22:23:36.603 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2024-01-21T22:23:36.603 INFO:teuthology.orchestra.run.smithi152.stdout: debug mgr = 20 2024-01-21T22:23:36.603 INFO:teuthology.orchestra.run.smithi152.stdout: debug mon = 20 2024-01-21T22:23:36.603 INFO:teuthology.orchestra.run.smithi152.stdout: debug auth = 20 2024-01-21T22:23:36.603 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min pgs per osd = 4 2024-01-21T22:23:36.603 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min bytes per osd = 10 2024-01-21T22:23:36.603 INFO:teuthology.orchestra.run.smithi152.stdout: mgr/telemetry/nag = false 2024-01-21T22:23:36.603 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.603 INFO:teuthology.orchestra.run.smithi152.stdout:[mon] 2024-01-21T22:23:36.603 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2024-01-21T22:23:36.603 INFO:teuthology.orchestra.run.smithi152.stdout: debug mon = 20 2024-01-21T22:23:36.604 INFO:teuthology.orchestra.run.smithi152.stdout: debug paxos = 20 2024-01-21T22:23:36.604 INFO:teuthology.orchestra.run.smithi152.stdout: debug auth = 20 2024-01-21T22:23:36.604 INFO:teuthology.orchestra.run.smithi152.stdout: mon data avail warn = 5 2024-01-21T22:23:36.604 INFO:teuthology.orchestra.run.smithi152.stdout: mon mgr mkfs grace = 240 2024-01-21T22:23:36.604 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min pgs per osd = 4 2024-01-21T22:23:36.604 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd reporter subtree level = osd 2024-01-21T22:23:36.604 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd prime pg temp = true 2024-01-21T22:23:36.604 INFO:teuthology.orchestra.run.smithi152.stdout: mon reweight min bytes per osd = 10 2024-01-21T22:23:36.604 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.604 INFO:teuthology.orchestra.run.smithi152.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-21T22:23:36.604 INFO:teuthology.orchestra.run.smithi152.stdout: auth mon ticket ttl = 660 # 11m 2024-01-21T22:23:36.604 INFO:teuthology.orchestra.run.smithi152.stdout: auth service ticket ttl = 240 # 4m 2024-01-21T22:23:36.604 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.604 INFO:teuthology.orchestra.run.smithi152.stdout: # don't complain about insecure global_id in the test suite 2024-01-21T22:23:36.605 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-21T22:23:36.605 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-21T22:23:36.605 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.605 INFO:teuthology.orchestra.run.smithi152.stdout: # 1m isn't quite enough 2024-01-21T22:23:36.605 INFO:teuthology.orchestra.run.smithi152.stdout: mon_down_mkfs_grace = 2m 2024-01-21T22:23:36.605 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.605 INFO:teuthology.orchestra.run.smithi152.stdout: mon_warn_on_filestore_osds = false 2024-01-21T22:23:36.605 INFO:teuthology.orchestra.run.smithi152.stdout: mon op complaint time = 120 2024-01-21T22:23:36.605 INFO:teuthology.orchestra.run.smithi152.stdout: mon osd crush smoke test = False 2024-01-21T22:23:36.605 INFO:teuthology.orchestra.run.smithi152.stdout: 2024-01-21T22:23:36.605 INFO:teuthology.orchestra.run.smithi152.stdout:[client] 2024-01-21T22:23:36.605 INFO:teuthology.orchestra.run.smithi152.stdout: rgw cache enabled = true 2024-01-21T22:23:36.605 INFO:teuthology.orchestra.run.smithi152.stdout: rgw enable ops log = true 2024-01-21T22:23:36.605 INFO:teuthology.orchestra.run.smithi152.stdout: rgw enable usage log = true 2024-01-21T22:23:36.605 INFO:teuthology.orchestra.run.smithi152.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-21T22:23:36.606 INFO:teuthology.orchestra.run.smithi152.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-21T22:23:36.606 INFO:teuthology.orchestra.run.smithi152.stdout: client mount timeout = 600 2024-01-21T22:23:36.606 INFO:teuthology.orchestra.run.smithi152.stdout: debug client = 20 2024-01-21T22:23:36.606 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2024-01-21T22:23:36.606 INFO:teuthology.orchestra.run.smithi152.stdout: rados mon op timeout = 900 2024-01-21T22:23:36.606 INFO:teuthology.orchestra.run.smithi152.stdout: rados osd op timeout = 900 2024-01-21T22:23:36.606 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.a] 2024-01-21T22:23:36.606 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.b] 2024-01-21T22:23:36.606 INFO:teuthology.orchestra.run.smithi152.stdout:[mon.c] 2024-01-21T22:23:36.606 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.a] 2024-01-21T22:23:36.606 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.c] 2024-01-21T22:23:36.606 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.e] 2024-01-21T22:23:36.606 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.b] 2024-01-21T22:23:36.606 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.d] 2024-01-21T22:23:36.607 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.f] 2024-01-21T22:23:36.607 INFO:teuthology.orchestra.run.smithi152.stdout:[mds] 2024-01-21T22:23:36.607 INFO:teuthology.orchestra.run.smithi152.stdout: debug mds = 20 2024-01-21T22:23:36.607 INFO:teuthology.orchestra.run.smithi152.stdout: debug mds balancer = 20 2024-01-21T22:23:36.607 INFO:teuthology.orchestra.run.smithi152.stdout: debug ms = 1 2024-01-21T22:23:36.607 INFO:teuthology.orchestra.run.smithi152.stdout: mds debug frag = True 2024-01-21T22:23:36.607 INFO:teuthology.orchestra.run.smithi152.stdout: mds debug scatterstat = True 2024-01-21T22:23:36.607 INFO:teuthology.orchestra.run.smithi152.stdout: mds op complaint time = 180 2024-01-21T22:23:36.607 INFO:teuthology.orchestra.run.smithi152.stdout: mds valgrind exit = True 2024-01-21T22:23:36.607 INFO:teuthology.orchestra.run.smithi152.stdout: mds verify scatter = True 2024-01-21T22:23:36.607 INFO:teuthology.orchestra.run.smithi152.stdout: osd op complaint time = 180 2024-01-21T22:23:36.607 INFO:teuthology.orchestra.run.smithi152.stdout: rados mon op timeout = 900 2024-01-21T22:23:36.607 INFO:teuthology.orchestra.run.smithi152.stdout: rados osd op timeout = 900 2024-01-21T22:23:36.696 INFO:teuthology.orchestra.run.smithi138.stdout:[global] 2024-01-21T22:23:36.696 INFO:teuthology.orchestra.run.smithi138.stdout: chdir = "" 2024-01-21T22:23:36.696 INFO:teuthology.orchestra.run.smithi138.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-01-21T22:23:36.696 INFO:teuthology.orchestra.run.smithi138.stdout: auth supported = cephx 2024-01-21T22:23:36.696 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.697 INFO:teuthology.orchestra.run.smithi138.stdout: filestore xattr use omap = true 2024-01-21T22:23:36.697 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.697 INFO:teuthology.orchestra.run.smithi138.stdout: mon clock drift allowed = 1.000 2024-01-21T22:23:36.697 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.697 INFO:teuthology.orchestra.run.smithi138.stdout: osd crush chooseleaf type = 0 2024-01-21T22:23:36.697 INFO:teuthology.orchestra.run.smithi138.stdout: auth debug = true 2024-01-21T22:23:36.697 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.697 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on old message = true 2024-01-21T22:23:36.697 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on bug = true 2024-01-21T22:23:36.697 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.697 INFO:teuthology.orchestra.run.smithi138.stdout: mon max pg per osd = 10000 # >= luminous 2024-01-21T22:23:36.697 INFO:teuthology.orchestra.run.smithi138.stdout: mon pg warn max object skew = 0 2024-01-21T22:23:36.697 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.697 INFO:teuthology.orchestra.run.smithi138.stdout: # disable pg_autoscaler by default for new pools 2024-01-21T22:23:36.698 INFO:teuthology.orchestra.run.smithi138.stdout: osd_pool_default_pg_autoscale_mode = off 2024-01-21T22:23:36.698 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.698 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default size = 2 2024-01-21T22:23:36.698 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.698 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow primary affinity = true 2024-01-21T22:23:36.698 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd allow pg remap = true 2024-01-21T22:23:36.698 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on legacy crush tunables = false 2024-01-21T22:23:36.698 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on crush straw calc version zero = false 2024-01-21T22:23:36.698 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on no sortbitwise = false 2024-01-21T22:23:36.698 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on osd down out interval zero = false 2024-01-21T22:23:36.698 INFO:teuthology.orchestra.run.smithi138.stdout: mon warn on too few osds = false 2024-01-21T22:23:36.698 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-01-21T22:23:36.698 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_pool_no_redundancy = false 2024-01-21T22:23:36.698 INFO:teuthology.orchestra.run.smithi138.stdout: mon_allow_pool_size_one = true 2024-01-21T22:23:36.698 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.699 INFO:teuthology.orchestra.run.smithi138.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-01-21T22:23:36.699 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.699 INFO:teuthology.orchestra.run.smithi138.stdout: osd default data pool replay window = 5 2024-01-21T22:23:36.699 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.699 INFO:teuthology.orchestra.run.smithi138.stdout: mon allow pool delete = true 2024-01-21T22:23:36.699 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.699 INFO:teuthology.orchestra.run.smithi138.stdout: mon cluster log file level = debug 2024-01-21T22:23:36.699 INFO:teuthology.orchestra.run.smithi138.stdout: debug asserts on shutdown = true 2024-01-21T22:23:36.699 INFO:teuthology.orchestra.run.smithi138.stdout: mon health detail to clog = false 2024-01-21T22:23:36.699 INFO:teuthology.orchestra.run.smithi138.stdout: mon host = "172.21.15.138,172.21.15.152,[v2:172.21.15.152:3301,v1:172.21.15.152:6790]" 2024-01-21T22:23:36.699 INFO:teuthology.orchestra.run.smithi138.stdout: mds beacon grace = 60 2024-01-21T22:23:36.699 INFO:teuthology.orchestra.run.smithi138.stdout: mds heartbeat grace = 60 2024-01-21T22:23:36.699 INFO:teuthology.orchestra.run.smithi138.stdout: ms die on skipped message = False 2024-01-21T22:23:36.699 INFO:teuthology.orchestra.run.smithi138.stdout: osd heartbeat grace = 60 2024-01-21T22:23:36.699 INFO:teuthology.orchestra.run.smithi138.stdout: fsid = d488d446-149f-48d0-89ca-b2830ae72846 2024-01-21T22:23:36.700 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.700 INFO:teuthology.orchestra.run.smithi138.stdout:[osd] 2024-01-21T22:23:36.700 INFO:teuthology.orchestra.run.smithi138.stdout: osd journal size = 100 2024-01-21T22:23:36.700 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.700 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub load threshold = 5.0 2024-01-21T22:23:36.700 INFO:teuthology.orchestra.run.smithi138.stdout: osd scrub max interval = 600 2024-01-21T22:23:36.700 INFO:teuthology.orchestra.run.smithi138.stdout: osd mclock profile = high_recovery_ops 2024-01-21T22:23:36.700 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.700 INFO:teuthology.orchestra.run.smithi138.stdout: osd recover clone overlap = true 2024-01-21T22:23:36.700 INFO:teuthology.orchestra.run.smithi138.stdout: osd recovery max chunk = 1048576 2024-01-21T22:23:36.700 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.700 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug shutdown = true 2024-01-21T22:23:36.700 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug op order = true 2024-01-21T22:23:36.700 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug verify stray on activate = true 2024-01-21T22:23:36.701 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug trim objects = true 2024-01-21T22:23:36.701 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.701 INFO:teuthology.orchestra.run.smithi138.stdout: osd open classes on start = true 2024-01-21T22:23:36.701 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug pg log writeout = true 2024-01-21T22:23:36.701 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.701 INFO:teuthology.orchestra.run.smithi138.stdout: osd deep scrub update digest min age = 30 2024-01-21T22:23:36.701 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.701 INFO:teuthology.orchestra.run.smithi138.stdout: osd map max advance = 10 2024-01-21T22:23:36.701 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.701 INFO:teuthology.orchestra.run.smithi138.stdout: journal zero on create = true 2024-01-21T22:23:36.701 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.701 INFO:teuthology.orchestra.run.smithi138.stdout: filestore ondisk finisher threads = 3 2024-01-21T22:23:36.701 INFO:teuthology.orchestra.run.smithi138.stdout: filestore apply finisher threads = 3 2024-01-21T22:23:36.701 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.702 INFO:teuthology.orchestra.run.smithi138.stdout: bdev debug aio = true 2024-01-21T22:23:36.702 INFO:teuthology.orchestra.run.smithi138.stdout: osd debug misdirected ops = true 2024-01-21T22:23:36.702 INFO:teuthology.orchestra.run.smithi138.stdout: bdev async discard = True 2024-01-21T22:23:36.702 INFO:teuthology.orchestra.run.smithi138.stdout: bdev enable discard = True 2024-01-21T22:23:36.702 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore allocator = bitmap 2024-01-21T22:23:36.702 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore block size = 96636764160 2024-01-21T22:23:36.702 INFO:teuthology.orchestra.run.smithi138.stdout: bluestore fsck on mount = True 2024-01-21T22:23:36.702 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluefs = 1/20 2024-01-21T22:23:36.702 INFO:teuthology.orchestra.run.smithi138.stdout: debug bluestore = 1/20 2024-01-21T22:23:36.702 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-01-21T22:23:36.702 INFO:teuthology.orchestra.run.smithi138.stdout: debug osd = 20 2024-01-21T22:23:36.702 INFO:teuthology.orchestra.run.smithi138.stdout: debug rocksdb = 4/10 2024-01-21T22:23:36.702 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd backfillfull_ratio = 0.85 2024-01-21T22:23:36.702 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd full ratio = 0.9 2024-01-21T22:23:36.702 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd nearfull ratio = 0.8 2024-01-21T22:23:36.702 INFO:teuthology.orchestra.run.smithi138.stdout: osd failsafe full ratio = 0.95 2024-01-21T22:23:36.703 INFO:teuthology.orchestra.run.smithi138.stdout: osd fast shutdown = False 2024-01-21T22:23:36.703 INFO:teuthology.orchestra.run.smithi138.stdout: osd objectstore = bluestore 2024-01-21T22:23:36.703 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2024-01-21T22:23:36.703 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.703 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr] 2024-01-21T22:23:36.703 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-01-21T22:23:36.703 INFO:teuthology.orchestra.run.smithi138.stdout: debug mgr = 20 2024-01-21T22:23:36.703 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-01-21T22:23:36.703 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-01-21T22:23:36.703 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-01-21T22:23:36.703 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-01-21T22:23:36.703 INFO:teuthology.orchestra.run.smithi138.stdout: mgr/telemetry/nag = false 2024-01-21T22:23:36.703 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.703 INFO:teuthology.orchestra.run.smithi138.stdout:[mon] 2024-01-21T22:23:36.703 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-01-21T22:23:36.704 INFO:teuthology.orchestra.run.smithi138.stdout: debug mon = 20 2024-01-21T22:23:36.704 INFO:teuthology.orchestra.run.smithi138.stdout: debug paxos = 20 2024-01-21T22:23:36.704 INFO:teuthology.orchestra.run.smithi138.stdout: debug auth = 20 2024-01-21T22:23:36.704 INFO:teuthology.orchestra.run.smithi138.stdout: mon data avail warn = 5 2024-01-21T22:23:36.704 INFO:teuthology.orchestra.run.smithi138.stdout: mon mgr mkfs grace = 240 2024-01-21T22:23:36.704 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min pgs per osd = 4 2024-01-21T22:23:36.704 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd reporter subtree level = osd 2024-01-21T22:23:36.704 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd prime pg temp = true 2024-01-21T22:23:36.704 INFO:teuthology.orchestra.run.smithi138.stdout: mon reweight min bytes per osd = 10 2024-01-21T22:23:36.704 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.704 INFO:teuthology.orchestra.run.smithi138.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-01-21T22:23:36.704 INFO:teuthology.orchestra.run.smithi138.stdout: auth mon ticket ttl = 660 # 11m 2024-01-21T22:23:36.704 INFO:teuthology.orchestra.run.smithi138.stdout: auth service ticket ttl = 240 # 4m 2024-01-21T22:23:36.704 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.704 INFO:teuthology.orchestra.run.smithi138.stdout: # don't complain about insecure global_id in the test suite 2024-01-21T22:23:36.705 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-01-21T22:23:36.705 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-01-21T22:23:36.705 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.705 INFO:teuthology.orchestra.run.smithi138.stdout: # 1m isn't quite enough 2024-01-21T22:23:36.705 INFO:teuthology.orchestra.run.smithi138.stdout: mon_down_mkfs_grace = 2m 2024-01-21T22:23:36.705 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.705 INFO:teuthology.orchestra.run.smithi138.stdout: mon_warn_on_filestore_osds = false 2024-01-21T22:23:36.705 INFO:teuthology.orchestra.run.smithi138.stdout: mon op complaint time = 120 2024-01-21T22:23:36.705 INFO:teuthology.orchestra.run.smithi138.stdout: mon osd crush smoke test = False 2024-01-21T22:23:36.705 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:23:36.705 INFO:teuthology.orchestra.run.smithi138.stdout:[client] 2024-01-21T22:23:36.705 INFO:teuthology.orchestra.run.smithi138.stdout: rgw cache enabled = true 2024-01-21T22:23:36.705 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable ops log = true 2024-01-21T22:23:36.705 INFO:teuthology.orchestra.run.smithi138.stdout: rgw enable usage log = true 2024-01-21T22:23:36.705 INFO:teuthology.orchestra.run.smithi138.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-01-21T22:23:36.706 INFO:teuthology.orchestra.run.smithi138.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-01-21T22:23:36.706 INFO:teuthology.orchestra.run.smithi138.stdout: client mount timeout = 600 2024-01-21T22:23:36.706 INFO:teuthology.orchestra.run.smithi138.stdout: debug client = 20 2024-01-21T22:23:36.706 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-01-21T22:23:36.706 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2024-01-21T22:23:36.706 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2024-01-21T22:23:36.706 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.a] 2024-01-21T22:23:36.706 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.b] 2024-01-21T22:23:36.706 INFO:teuthology.orchestra.run.smithi138.stdout:[mon.c] 2024-01-21T22:23:36.706 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a] 2024-01-21T22:23:36.706 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.c] 2024-01-21T22:23:36.706 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.e] 2024-01-21T22:23:36.706 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.b] 2024-01-21T22:23:36.706 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.d] 2024-01-21T22:23:36.706 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.f] 2024-01-21T22:23:36.706 INFO:teuthology.orchestra.run.smithi138.stdout:[mds] 2024-01-21T22:23:36.707 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds = 20 2024-01-21T22:23:36.707 INFO:teuthology.orchestra.run.smithi138.stdout: debug mds balancer = 20 2024-01-21T22:23:36.707 INFO:teuthology.orchestra.run.smithi138.stdout: debug ms = 1 2024-01-21T22:23:36.707 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug frag = True 2024-01-21T22:23:36.707 INFO:teuthology.orchestra.run.smithi138.stdout: mds debug scatterstat = True 2024-01-21T22:23:36.707 INFO:teuthology.orchestra.run.smithi138.stdout: mds op complaint time = 180 2024-01-21T22:23:36.707 INFO:teuthology.orchestra.run.smithi138.stdout: mds valgrind exit = True 2024-01-21T22:23:36.707 INFO:teuthology.orchestra.run.smithi138.stdout: mds verify scatter = True 2024-01-21T22:23:36.707 INFO:teuthology.orchestra.run.smithi138.stdout: osd op complaint time = 180 2024-01-21T22:23:36.707 INFO:teuthology.orchestra.run.smithi138.stdout: rados mon op timeout = 900 2024-01-21T22:23:36.707 INFO:teuthology.orchestra.run.smithi138.stdout: rados osd op timeout = 900 2024-01-21T22:23:36.742 INFO:tasks.ceph:Creating admin key on mon.a... 2024-01-21T22:23:36.742 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-01-21T22:23:36.886 INFO:tasks.ceph:Copying monmap to all nodes... 2024-01-21T22:23:36.887 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:23:36.887 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-01-21T22:23:36.936 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:23:36.936 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-01-21T22:23:37.032 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi138.front.sepia.ceph.com 2024-01-21T22:23:37.032 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:23:37.032 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-21T22:23:37.033 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-21T22:23:37.178 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:23:37.178 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-21T22:23:37.274 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi152.front.sepia.ceph.com 2024-01-21T22:23:37.274 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-21T22:23:37.274 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd of=/etc/ceph/ceph.keyring 2024-01-21T22:23:37.274 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-01-21T22:23:37.316 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-21T22:23:37.317 DEBUG:teuthology.orchestra.run.smithi152:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-01-21T22:23:37.373 INFO:tasks.ceph:Setting up mon nodes... 2024-01-21T22:23:37.373 INFO:tasks.ceph:Setting up mgr nodes... 2024-01-21T22:23:37.373 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-01-21T22:23:37.559 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-01-21T22:23:37.569 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-01-21T22:23:37.637 INFO:teuthology.orchestra.run.smithi152.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-01-21T22:23:37.640 INFO:tasks.ceph:Setting up mds nodes... 2024-01-21T22:23:37.640 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-01-21T22:23:37.823 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-01-21T22:23:37.833 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-01-21T22:23:37.923 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-01-21T22:23:38.103 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-01-21T22:23:38.112 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-01-21T22:23:38.208 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-01-21T22:23:38.373 INFO:teuthology.orchestra.run.smithi138.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-01-21T22:23:38.382 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-01-21T22:23:38.476 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-01-21T22:23:38.538 INFO:teuthology.orchestra.run.smithi152.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-01-21T22:23:38.541 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-01-21T22:23:38.569 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-01-21T22:23:38.666 INFO:teuthology.orchestra.run.smithi152.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-01-21T22:23:38.669 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-01-21T22:23:38.695 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-01-21T22:23:38.791 INFO:teuthology.orchestra.run.smithi152.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-01-21T22:23:38.793 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-01-21T22:23:38.820 INFO:tasks.ceph_client:Setting up client nodes... 2024-01-21T22:23:38.823 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-01-21T22:23:38.947 INFO:teuthology.orchestra.run.smithi138.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-01-21T22:23:39.003 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-01-21T22:23:39.003 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi138.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi152.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-01-21T22:23:39.004 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-01-21T22:23:39.101 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-21T22:23:39.101 INFO:tasks.ceph:role: osd.0 2024-01-21T22:23:39.101 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com 2024-01-21T22:23:39.101 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-21T22:23:39.201 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-21T22:23:39.201 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-21T22:23:39.201 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-21T22:23:39.201 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-21T22:23:39.201 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-21T22:23:39.201 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-01-21T22:23:39.201 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-21T22:23:39.201 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-21T22:23:39.201 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-21T22:23:39.202 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-21T22:23:39.711 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-01-21T22:23:39.717 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-01-21T22:23:39.717 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-01-21T22:23:39.900 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-01-21T22:23:39.996 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-01-21T22:23:40.073 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-21T22:23:40.073 INFO:tasks.ceph:role: osd.1 2024-01-21T22:23:40.073 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com 2024-01-21T22:23:40.073 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-21T22:23:40.159 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-21T22:23:40.159 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-21T22:23:40.159 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-21T22:23:40.159 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-21T22:23:40.160 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-21T22:23:40.160 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-01-21T22:23:40.160 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-21T22:23:40.160 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-21T22:23:40.160 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-21T22:23:40.160 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-21T22:23:40.617 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-01-21T22:23:40.624 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-01-21T22:23:40.624 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-01-21T22:23:40.740 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-01-21T22:23:40.827 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-01-21T22:23:40.903 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-21T22:23:40.903 INFO:tasks.ceph:role: osd.2 2024-01-21T22:23:40.903 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi138.front.sepia.ceph.com 2024-01-21T22:23:40.904 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-21T22:23:40.993 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-21T22:23:40.993 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-21T22:23:40.993 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-21T22:23:40.993 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-21T22:23:40.993 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-21T22:23:40.993 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-01-21T22:23:40.993 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-21T22:23:40.993 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-21T22:23:40.994 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-21T22:23:40.994 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-21T22:23:41.508 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-01-21T22:23:41.513 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-01-21T22:23:41.514 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-01-21T22:23:41.670 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-01-21T22:23:41.764 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-01-21T22:23:41.845 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-01-21T22:23:41.845 INFO:tasks.ceph:role: osd.3 2024-01-21T22:23:41.845 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi138.front.sepia.ceph.com 2024-01-21T22:23:41.845 DEBUG:teuthology.orchestra.run.smithi138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-01-21T22:23:41.936 INFO:teuthology.orchestra.run.smithi138.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-01-21T22:23:41.936 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-21T22:23:41.936 INFO:teuthology.orchestra.run.smithi138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-21T22:23:41.936 INFO:teuthology.orchestra.run.smithi138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-21T22:23:41.937 INFO:teuthology.orchestra.run.smithi138.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-21T22:23:41.937 INFO:teuthology.orchestra.run.smithi138.stdout: = sunit=0 swidth=0 blks 2024-01-21T22:23:41.937 INFO:teuthology.orchestra.run.smithi138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-21T22:23:41.937 INFO:teuthology.orchestra.run.smithi138.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-21T22:23:41.937 INFO:teuthology.orchestra.run.smithi138.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-21T22:23:41.937 INFO:teuthology.orchestra.run.smithi138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-21T22:23:42.393 INFO:teuthology.orchestra.run.smithi138.stdout:Discarding blocks...Done. 2024-01-21T22:23:42.399 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi138.front.sepia.ceph.com -o noatime 2024-01-21T22:23:42.399 DEBUG:teuthology.orchestra.run.smithi138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-01-21T22:23:42.513 DEBUG:teuthology.orchestra.run.smithi138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-01-21T22:23:42.594 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-21T22:23:42.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:42.729+0000 7f07c6815640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-01-21T22:23:42.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:42.729+0000 7f07c6815640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-01-21T22:23:42.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:42.730+0000 7f07c6815640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-21T22:23:42.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:42.730+0000 7f07c6815640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-21T22:23:42.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:42.730+0000 7f07c6815640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-01-21T22:23:42.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:42.733+0000 7f07c6815640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-01-21T22:23:45.899 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-01-21T22:23:45.999 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-21T22:23:46.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:46.125+0000 7f7c3def5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-01-21T22:23:46.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:46.126+0000 7f7c3def5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-01-21T22:23:46.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:46.126+0000 7f7c3def5640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-21T22:23:46.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:46.126+0000 7f7c3def5640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-21T22:23:46.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:46.126+0000 7f7c3def5640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-01-21T22:23:46.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:46.128+0000 7f7c3def5640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-01-21T22:23:49.300 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-01-21T22:23:49.396 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-21T22:23:49.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:49.525+0000 7f4d38c1b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-01-21T22:23:49.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:49.526+0000 7f4d38c1b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-01-21T22:23:49.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:49.526+0000 7f4d38c1b640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-21T22:23:49.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:49.526+0000 7f4d38c1b640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-21T22:23:49.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:49.526+0000 7f4d38c1b640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-01-21T22:23:49.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:49.528+0000 7f4d38c1b640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-01-21T22:23:52.457 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-01-21T22:23:52.551 DEBUG:teuthology.orchestra.run.smithi138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-21T22:23:52.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:52.684+0000 7f27d7011640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-01-21T22:23:52.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:52.685+0000 7f27d7011640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-01-21T22:23:52.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:52.685+0000 7f27d7011640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-21T22:23:52.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:52.685+0000 7f27d7011640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-21T22:23:52.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:52.685+0000 7f27d7011640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-01-21T22:23:52.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:23:52.687+0000 7f27d7011640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-01-21T22:23:55.863 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-01-21T22:23:55.962 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-01-21T22:23:55.995 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-21T22:23:55.996 INFO:tasks.ceph:role: osd.4 2024-01-21T22:23:55.996 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi152.front.sepia.ceph.com 2024-01-21T22:23:55.996 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-01-21T22:23:56.073 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-01-21T22:23:56.073 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-21T22:23:56.073 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-21T22:23:56.073 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-21T22:23:56.073 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-21T22:23:56.073 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2024-01-21T22:23:56.073 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-21T22:23:56.074 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-21T22:23:56.074 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-21T22:23:56.074 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-21T22:23:56.598 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2024-01-21T22:23:56.600 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2024-01-21T22:23:56.601 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-01-21T22:23:56.735 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-01-21T22:23:56.772 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-01-21T22:23:56.842 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-21T22:23:56.842 INFO:tasks.ceph:role: osd.5 2024-01-21T22:23:56.842 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi152.front.sepia.ceph.com 2024-01-21T22:23:56.842 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-01-21T22:23:56.919 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-01-21T22:23:56.919 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-21T22:23:56.919 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-21T22:23:56.919 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-21T22:23:56.919 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-21T22:23:56.919 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2024-01-21T22:23:56.919 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-21T22:23:56.919 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-21T22:23:56.919 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-21T22:23:56.919 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-21T22:23:57.364 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2024-01-21T22:23:57.366 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2024-01-21T22:23:57.367 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-01-21T22:23:57.472 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-01-21T22:23:57.505 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-01-21T22:23:57.573 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-21T22:23:57.573 INFO:tasks.ceph:role: osd.6 2024-01-21T22:23:57.573 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi152.front.sepia.ceph.com 2024-01-21T22:23:57.574 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-01-21T22:23:57.649 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-01-21T22:23:57.649 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-21T22:23:57.649 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-21T22:23:57.649 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-21T22:23:57.649 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-21T22:23:57.650 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2024-01-21T22:23:57.650 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-21T22:23:57.650 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-21T22:23:57.650 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-21T22:23:57.650 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-21T22:23:58.141 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2024-01-21T22:23:58.143 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2024-01-21T22:23:58.143 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-01-21T22:23:58.297 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-01-21T22:23:58.333 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-01-21T22:23:58.401 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-01-21T22:23:58.401 INFO:tasks.ceph:role: osd.7 2024-01-21T22:23:58.401 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi152.front.sepia.ceph.com 2024-01-21T22:23:58.402 DEBUG:teuthology.orchestra.run.smithi152:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-01-21T22:23:58.477 INFO:teuthology.orchestra.run.smithi152.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-01-21T22:23:58.478 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 attr=2, projid32bit=1 2024-01-21T22:23:58.478 INFO:teuthology.orchestra.run.smithi152.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-01-21T22:23:58.478 INFO:teuthology.orchestra.run.smithi152.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-01-21T22:23:58.478 INFO:teuthology.orchestra.run.smithi152.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-01-21T22:23:58.478 INFO:teuthology.orchestra.run.smithi152.stdout: = sunit=0 swidth=0 blks 2024-01-21T22:23:58.478 INFO:teuthology.orchestra.run.smithi152.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-01-21T22:23:58.478 INFO:teuthology.orchestra.run.smithi152.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-01-21T22:23:58.478 INFO:teuthology.orchestra.run.smithi152.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-01-21T22:23:58.478 INFO:teuthology.orchestra.run.smithi152.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-01-21T22:23:58.993 INFO:teuthology.orchestra.run.smithi152.stdout:Discarding blocks...Done. 2024-01-21T22:23:58.995 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi152.front.sepia.ceph.com -o noatime 2024-01-21T22:23:58.995 DEBUG:teuthology.orchestra.run.smithi152:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-01-21T22:23:59.128 DEBUG:teuthology.orchestra.run.smithi152:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-01-21T22:23:59.164 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-21T22:23:59.268 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:23:59.266+0000 7fde1c815640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-01-21T22:23:59.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:23:59.266+0000 7fde1c815640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-01-21T22:23:59.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:23:59.266+0000 7fde1c815640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-21T22:23:59.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:23:59.266+0000 7fde1c815640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-21T22:23:59.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:23:59.266+0000 7fde1c815640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-01-21T22:23:59.269 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:23:59.267+0000 7fde1c815640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-01-21T22:24:02.121 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-01-21T22:24:02.174 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-21T22:24:02.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:02.261+0000 7fe824c1b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-01-21T22:24:02.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:02.261+0000 7fe824c1b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-01-21T22:24:02.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:02.261+0000 7fe824c1b640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-21T22:24:02.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:02.261+0000 7fe824c1b640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-21T22:24:02.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:02.261+0000 7fe824c1b640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-01-21T22:24:02.264 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:02.261+0000 7fe824c1b640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-01-21T22:24:04.131 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:24:04.138 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:24:05.114 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-01-21T22:24:05.142 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-21T22:24:05.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:05.226+0000 7fb6b8a19640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-01-21T22:24:05.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:05.227+0000 7fb6b8a19640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-01-21T22:24:05.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:05.227+0000 7fb6b8a19640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-21T22:24:05.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:05.227+0000 7fb6b8a19640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-21T22:24:05.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:05.227+0000 7fb6b8a19640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-01-21T22:24:05.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:05.227+0000 7fb6b8a19640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-01-21T22:24:08.076 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-01-21T22:24:08.107 DEBUG:teuthology.orchestra.run.smithi152:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-01-21T22:24:08.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:08.193+0000 7f1b46973640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-01-21T22:24:08.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:08.193+0000 7f1b46973640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-01-21T22:24:08.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:08.194+0000 7f1b46973640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-21T22:24:08.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:08.194+0000 7f1b46973640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-21T22:24:08.196 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:08.194+0000 7f1b46973640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-01-21T22:24:08.197 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:08.194+0000 7f1b46973640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-01-21T22:24:11.045 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-01-21T22:24:11.074 INFO:tasks.ceph:Reading keys from all nodes... 2024-01-21T22:24:11.074 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:24:11.074 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-01-21T22:24:11.174 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:24:11.174 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-01-21T22:24:11.267 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:24:11.267 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-01-21T22:24:11.363 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:24:11.363 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-01-21T22:24:11.454 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:24:11.454 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-01-21T22:24:11.548 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:24:11.548 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-01-21T22:24:11.641 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:24:11.641 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-01-21T22:24:11.732 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:24:11.732 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-01-21T22:24:11.826 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-21T22:24:11.826 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-01-21T22:24:11.857 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-21T22:24:11.857 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-01-21T22:24:11.929 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-21T22:24:11.929 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-01-21T22:24:11.998 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-21T22:24:11.999 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-01-21T22:24:12.068 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-21T22:24:12.068 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-01-21T22:24:12.138 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-21T22:24:12.138 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-01-21T22:24:12.206 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-21T22:24:12.206 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-01-21T22:24:12.274 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-21T22:24:12.274 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-01-21T22:24:12.341 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:24:12.342 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-01-21T22:24:12.397 INFO:tasks.ceph:Adding keys to all mons... 2024-01-21T22:24:12.397 DEBUG:teuthology.orchestra.run.smithi138:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-21T22:24:12.402 DEBUG:teuthology.orchestra.run.smithi152:> sudo tee -a /etc/ceph/ceph.keyring 2024-01-21T22:24:12.427 INFO:teuthology.orchestra.run.smithi152.stdout:[mgr.y] 2024-01-21T22:24:12.428 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBpma1l41BJIRAADI38Y3MMXfcBS2Sv2nbQlw== 2024-01-21T22:24:12.428 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.a] 2024-01-21T22:24:12.428 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBpma1laj4SMRAA94X8DmyZhBAzmKwUVdbOFA== 2024-01-21T22:24:12.428 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.c] 2024-01-21T22:24:12.428 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBqma1lElAfBhAAEMcE655GK0c/8sE3IERMtA== 2024-01-21T22:24:12.428 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.e] 2024-01-21T22:24:12.428 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBqma1lSRo8FhAA4fcd3P3jk7MPnZZOIYDbnA== 2024-01-21T22:24:12.428 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.0] 2024-01-21T22:24:12.428 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBuma1l6gqQKxAANbJIevAgAwVB0gtuOLwWHw== 2024-01-21T22:24:12.428 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.1] 2024-01-21T22:24:12.428 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQByma1lFsKVBxAAO7LXy4lyc1t9iSTb69LEow== 2024-01-21T22:24:12.428 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.2] 2024-01-21T22:24:12.428 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQB1ma1lXIhtHxAAP+SCGJYWcmC8xl0ejCO5ig== 2024-01-21T22:24:12.429 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.3] 2024-01-21T22:24:12.429 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQB4ma1l6NzZKBAAyP6EkKmp6Fwe3qO/Ge51JA== 2024-01-21T22:24:12.429 INFO:teuthology.orchestra.run.smithi152.stdout:[mgr.x] 2024-01-21T22:24:12.429 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBpma1lIjv1JRAAWIB3FXhzbfTje45PKmn1pQ== 2024-01-21T22:24:12.429 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.b] 2024-01-21T22:24:12.429 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBqma1l4x4TIBAAFF9whEynO7xwTN3PmKfykw== 2024-01-21T22:24:12.429 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.d] 2024-01-21T22:24:12.429 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBqma1l29e2JxAAxZEPtubn1o+VNaDqmFToSw== 2024-01-21T22:24:12.429 INFO:teuthology.orchestra.run.smithi152.stdout:[mds.f] 2024-01-21T22:24:12.429 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBqma1lPSkiLxAAvWjLVdLVPCZBfFMFv8EBew== 2024-01-21T22:24:12.429 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.4] 2024-01-21T22:24:12.429 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQB/ma1lPnT4DxAAEORieUtTZChzzrPDyc/5mg== 2024-01-21T22:24:12.429 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.5] 2024-01-21T22:24:12.429 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCCma1l5EWpDxAA1WLG6RRRNYi7jsj2eElgPw== 2024-01-21T22:24:12.429 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.6] 2024-01-21T22:24:12.429 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCFma1lwL6dDRAA4f/mVfg9PgqHNSEIck1TDg== 2024-01-21T22:24:12.430 INFO:teuthology.orchestra.run.smithi152.stdout:[osd.7] 2024-01-21T22:24:12.430 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQCIma1lrK+kCxAAq7r21/9AEmuvPmXvgAiR8w== 2024-01-21T22:24:12.430 INFO:teuthology.orchestra.run.smithi152.stdout:[client.0] 2024-01-21T22:24:12.430 INFO:teuthology.orchestra.run.smithi152.stdout: key = AQBqma1lHz92OBAATzBLawyC8pm85cjE5v3ymQ== 2024-01-21T22:24:12.475 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.y] 2024-01-21T22:24:12.475 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQBpma1l41BJIRAADI38Y3MMXfcBS2Sv2nbQlw== 2024-01-21T22:24:12.475 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a] 2024-01-21T22:24:12.475 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQBpma1laj4SMRAA94X8DmyZhBAzmKwUVdbOFA== 2024-01-21T22:24:12.475 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.c] 2024-01-21T22:24:12.475 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQBqma1lElAfBhAAEMcE655GK0c/8sE3IERMtA== 2024-01-21T22:24:12.475 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.e] 2024-01-21T22:24:12.476 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQBqma1lSRo8FhAA4fcd3P3jk7MPnZZOIYDbnA== 2024-01-21T22:24:12.476 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.0] 2024-01-21T22:24:12.476 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQBuma1l6gqQKxAANbJIevAgAwVB0gtuOLwWHw== 2024-01-21T22:24:12.476 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.1] 2024-01-21T22:24:12.476 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQByma1lFsKVBxAAO7LXy4lyc1t9iSTb69LEow== 2024-01-21T22:24:12.476 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.2] 2024-01-21T22:24:12.476 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB1ma1lXIhtHxAAP+SCGJYWcmC8xl0ejCO5ig== 2024-01-21T22:24:12.476 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.3] 2024-01-21T22:24:12.476 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB4ma1l6NzZKBAAyP6EkKmp6Fwe3qO/Ge51JA== 2024-01-21T22:24:12.476 INFO:teuthology.orchestra.run.smithi138.stdout:[mgr.x] 2024-01-21T22:24:12.476 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQBpma1lIjv1JRAAWIB3FXhzbfTje45PKmn1pQ== 2024-01-21T22:24:12.476 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.b] 2024-01-21T22:24:12.477 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQBqma1l4x4TIBAAFF9whEynO7xwTN3PmKfykw== 2024-01-21T22:24:12.477 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.d] 2024-01-21T22:24:12.477 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQBqma1l29e2JxAAxZEPtubn1o+VNaDqmFToSw== 2024-01-21T22:24:12.477 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.f] 2024-01-21T22:24:12.477 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQBqma1lPSkiLxAAvWjLVdLVPCZBfFMFv8EBew== 2024-01-21T22:24:12.477 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.4] 2024-01-21T22:24:12.477 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQB/ma1lPnT4DxAAEORieUtTZChzzrPDyc/5mg== 2024-01-21T22:24:12.477 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.5] 2024-01-21T22:24:12.477 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCCma1l5EWpDxAA1WLG6RRRNYi7jsj2eElgPw== 2024-01-21T22:24:12.477 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.6] 2024-01-21T22:24:12.477 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCFma1lwL6dDRAA4f/mVfg9PgqHNSEIck1TDg== 2024-01-21T22:24:12.477 INFO:teuthology.orchestra.run.smithi138.stdout:[osd.7] 2024-01-21T22:24:12.477 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCIma1lrK+kCxAAq7r21/9AEmuvPmXvgAiR8w== 2024-01-21T22:24:12.477 INFO:teuthology.orchestra.run.smithi138.stdout:[client.0] 2024-01-21T22:24:12.477 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQBqma1lHz92OBAATzBLawyC8pm85cjE5v3ymQ== 2024-01-21T22:24:12.478 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-21T22:24:12.527 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-21T22:24:12.670 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-21T22:24:12.677 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-21T22:24:12.815 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-21T22:24:12.820 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-21T22:24:12.945 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-21T22:24:12.951 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-21T22:24:13.085 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-21T22:24:13.089 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-21T22:24:13.209 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-21T22:24:13.214 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-21T22:24:13.345 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-21T22:24:13.349 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-21T22:24:13.483 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-21T22:24:13.488 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-21T22:24:13.623 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-21T22:24:13.628 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-01-21T22:24:13.751 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-21T22:24:13.756 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-21T22:24:13.898 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-21T22:24:13.905 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-21T22:24:14.042 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-21T22:24:14.050 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-01-21T22:24:14.185 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-21T22:24:14.189 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-21T22:24:14.316 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-21T22:24:14.321 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-21T22:24:14.451 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-21T22:24:14.459 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-21T22:24:14.591 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-21T22:24:14.596 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-01-21T22:24:14.723 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-21T22:24:14.729 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-01-21T22:24:14.866 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-01-21T22:24:14.866 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-01-21T22:24:14.959 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-21T22:24:15.112 DEBUG:teuthology.orchestra.run.smithi138:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-01-21T22:24:15.195 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-01-21T22:24:15.222 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-21T22:24:15.319 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-01-21T22:24:15.345 DEBUG:teuthology.orchestra.run.smithi152:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-01-21T22:24:15.411 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-01-21T22:24:15.509 DEBUG:teuthology.orchestra.run.smithi152:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-01-21T22:24:15.536 DEBUG:teuthology.orchestra.run.smithi138:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-21T22:24:15.543 DEBUG:teuthology.orchestra.run.smithi152:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-01-21T22:24:15.591 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-01-21T22:24:15.591 DEBUG:tasks.ceph_manager:running mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.a.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2024-01-21T22:24:15.592 INFO:tasks.ceph.mon.a:Restarting daemon 2024-01-21T22:24:15.592 DEBUG:teuthology.orchestra.run.smithi138:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2024-01-21T22:24:15.599 INFO:tasks.ceph.mon.a:Started 2024-01-21T22:24:15.599 DEBUG:tasks.ceph_manager:running mon.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.b.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2024-01-21T22:24:15.599 INFO:tasks.ceph.mon.b:Restarting daemon 2024-01-21T22:24:15.599 DEBUG:teuthology.orchestra.run.smithi152:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2024-01-21T22:24:15.634 INFO:tasks.ceph.mon.b:Started 2024-01-21T22:24:15.634 DEBUG:tasks.ceph_manager:running mon.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.c.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2024-01-21T22:24:15.634 INFO:tasks.ceph.mon.c:Restarting daemon 2024-01-21T22:24:15.634 DEBUG:teuthology.orchestra.run.smithi152:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2024-01-21T22:24:15.636 INFO:tasks.ceph.mon.c:Started 2024-01-21T22:24:15.636 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-01-21T22:24:15.636 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-01-21T22:24:15.637 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2024-01-21T22:24:15.642 INFO:tasks.ceph.mgr.y:Started 2024-01-21T22:24:15.642 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-01-21T22:24:15.642 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2024-01-21T22:24:15.645 INFO:tasks.ceph.mgr.x:Started 2024-01-21T22:24:15.645 INFO:tasks.ceph:Setting crush tunables to default 2024-01-21T22:24:15.645 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph osd crush tunables default 2024-01-21T22:24:15.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:15.816+0000 7f01b6600640 1 Processor -- start 2024-01-21T22:24:15.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:15.816+0000 7f01b6600640 1 -- start start 2024-01-21T22:24:15.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:15.817+0000 7f01b6600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f01b0153c60 0x7f01b00c7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:15.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:15.817+0000 7f01b6600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f01b00c9bd0 0x7f01b00c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:15.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:15.817+0000 7f01b6600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b00c7290 con 0x7f01b00c6900 2024-01-21T22:24:15.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:15.817+0000 7f01b6600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b015a570 con 0x7f01b0153c60 2024-01-21T22:24:15.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:15.817+0000 7f01b6600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b00cb640 con 0x7f01b00c9bd0 2024-01-21T22:24:15.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:15.817+0000 7f01af400640 1 -- >> v2:172.21.15.138:3300/0 conn(0x7f01b0153c60 msgr2=0x7f01b00c7720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.138:3300/0 2024-01-21T22:24:15.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:15.817+0000 7f01a7e00640 1 -- >> v2:172.21.15.152:3300/0 conn(0x7f01b00c9bd0 msgr2=0x7f01b00c7c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.152:3300/0 2024-01-21T22:24:15.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:15.817+0000 7f01af400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f01b0153c60 0x7f01b00c7720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-21T22:24:15.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:15.817+0000 7f01afe00640 1 -- >> v1:172.21.15.138:6789/0 conn(0x7f01b00c6900 legacy=0x7f01b00ca8c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.138:6789/0 2024-01-21T22:24:15.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:15.817+0000 7f01a7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f01b00c9bd0 0x7f01b00c7c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-21T22:24:16.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:16.018+0000 7f01afe00640 1 -- >> v1:172.21.15.138:6789/0 conn(0x7f01b00c6900 legacy=0x7f01b00ca8c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.138:6789/0 2024-01-21T22:24:16.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:16.018+0000 7f01af400640 1 -- >> v2:172.21.15.138:3300/0 conn(0x7f01b0153c60 msgr2=0x7f01b00c7720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.138:3300/0 2024-01-21T22:24:16.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:16.018+0000 7f01a7e00640 1 -- >> v2:172.21.15.152:3300/0 conn(0x7f01b00c9bd0 msgr2=0x7f01b00c7c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.152:3300/0 2024-01-21T22:24:16.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:16.018+0000 7f01af400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f01b0153c60 0x7f01b00c7720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-21T22:24:16.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:16.018+0000 7f01a7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f01b00c9bd0 0x7f01b00c7c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-21T22:24:16.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:16.419+0000 7f01afe00640 1 -- >> v1:172.21.15.138:6789/0 conn(0x7f01b00c6900 legacy=0x7f01b00ca8c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.138:6789/0 2024-01-21T22:24:16.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:16.419+0000 7f01af400640 1 -- >> v2:172.21.15.138:3300/0 conn(0x7f01b0153c60 msgr2=0x7f01b00c7720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.138:3300/0 2024-01-21T22:24:16.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:16.419+0000 7f01a7e00640 1 -- >> v2:172.21.15.152:3300/0 conn(0x7f01b00c9bd0 msgr2=0x7f01b00c7c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.152:3300/0 2024-01-21T22:24:16.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:16.419+0000 7f01af400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f01b0153c60 0x7f01b00c7720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-21T22:24:16.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:16.419+0000 7f01a7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f01b00c9bd0 0x7f01b00c7c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-21T22:24:17.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:17.220+0000 7f01afe00640 1 -- >> v1:172.21.15.138:6789/0 conn(0x7f01b00c6900 legacy=0x7f01b00ca8c0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.138:6789/0 2024-01-21T22:24:17.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:17.220+0000 7f01af400640 1 -- >> v2:172.21.15.138:3300/0 conn(0x7f01b0153c60 msgr2=0x7f01b00c7720 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.138:3300/0 2024-01-21T22:24:17.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:17.220+0000 7f01a7e00640 1 -- >> v2:172.21.15.152:3300/0 conn(0x7f01b00c9bd0 msgr2=0x7f01b00c7c60 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.152:3300/0 2024-01-21T22:24:17.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:17.220+0000 7f01af400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f01b0153c60 0x7f01b00c7720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-21T22:24:17.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:17.220+0000 7f01a7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f01b00c9bd0 0x7f01b00c7c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-21T22:24:18.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:18.817+0000 7f01ad600640 1 -- >> v2:172.21.15.152:3300/0 conn(0x7f01b00c9bd0 msgr2=0x7f01b00c7c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-21T22:24:18.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:18.817+0000 7f01ad600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f01b00c9bd0 0x7f01b00c7c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:18.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:18.817+0000 7f01ad600640 1 -- >> v2:172.21.15.138:3300/0 conn(0x7f01b0153c60 msgr2=0x7f01b00c7720 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-21T22:24:18.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:18.817+0000 7f01ad600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f01b0153c60 0x7f01b00c7720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:18.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:18.817+0000 7f01ad600640 1 -- >> v1:172.21.15.138:6789/0 conn(0x7f01b00c6900 legacy=0x7f01b00ca8c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-21T22:24:18.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:18.817+0000 7f01ad600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0188003690 0x7f0188005ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:18.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:18.817+0000 7f01ad600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b00ca110 con 0x7f0188006120 2024-01-21T22:24:18.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:18.817+0000 7f01ad600640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b015a570 con 0x7f0188003690 2024-01-21T22:24:18.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:18.818+0000 7f01a7e00640 1 -- >> v1:172.21.15.152:6789/0 conn(0x7f0188006120 legacy=0x7f01880085e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.152:6789/0 2024-01-21T22:24:18.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:18.818+0000 7f01af400640 1 -- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0188003690 msgr2=0x7f0188005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.152:3301/0 2024-01-21T22:24:18.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:18.818+0000 7f01af400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0188003690 0x7f0188005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-21T22:24:19.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:19.019+0000 7f01a7e00640 1 -- >> v1:172.21.15.152:6789/0 conn(0x7f0188006120 legacy=0x7f01880085e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.152:6789/0 2024-01-21T22:24:19.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:19.019+0000 7f01af400640 1 -- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0188003690 msgr2=0x7f0188005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.152:3301/0 2024-01-21T22:24:19.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:19.019+0000 7f01af400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0188003690 0x7f0188005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-21T22:24:19.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:19.419+0000 7f01a7e00640 1 -- >> v1:172.21.15.152:6789/0 conn(0x7f0188006120 legacy=0x7f01880085e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.152:6789/0 2024-01-21T22:24:19.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:19.419+0000 7f01af400640 1 -- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0188003690 msgr2=0x7f0188005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.152:3301/0 2024-01-21T22:24:19.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:19.419+0000 7f01af400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0188003690 0x7f0188005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-21T22:24:20.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:20.220+0000 7f01af400640 1 -- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0188003690 msgr2=0x7f0188005ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.152:3301/0 2024-01-21T22:24:20.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:20.220+0000 7f01a7e00640 1 -- >> v1:172.21.15.152:6789/0 conn(0x7f0188006120 legacy=0x7f01880085e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.152:6789/0 2024-01-21T22:24:20.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:20.220+0000 7f01af400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0188003690 0x7f0188005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-21T22:24:21.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:21.817+0000 7f01ad600640 1 -- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0188003690 msgr2=0x7f0188005ae0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-21T22:24:21.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:21.817+0000 7f01ad600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0188003690 0x7f0188005ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:21.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:21.817+0000 7f01ad600640 1 -- >> v1:172.21.15.152:6789/0 conn(0x7f0188006120 legacy=0x7f01880085e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-21T22:24:21.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:21.817+0000 7f01afe00640 1 -- reap_dead start 2024-01-21T22:24:21.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:21.817+0000 7f01ad600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f018800aa40 0x7f018800e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:21.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:21.817+0000 7f01ad600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0188008d10 0x7f01880085e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:21.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:21.817+0000 7f01ad600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b00c7290 con 0x7f0188009eb0 2024-01-21T22:24:21.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:21.817+0000 7f01ad600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b015a570 con 0x7f018800aa40 2024-01-21T22:24:21.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:21.817+0000 7f01ad600640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b00cb640 con 0x7f0188008d10 2024-01-21T22:24:21.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:21.817+0000 7f01afe00640 1 -- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0188008d10 msgr2=0x7f01880085e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.152:3301/0 2024-01-21T22:24:21.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:21.817+0000 7f01afe00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0188008d10 0x7f01880085e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-21T22:24:21.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:21.817+0000 7f01a7e00640 1 -- >> v2:172.21.15.152:3300/0 conn(0x7f018800aa40 msgr2=0x7f018800e0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.152:3300/0 2024-01-21T22:24:21.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:21.817+0000 7f01af400640 1 -- >> v1:172.21.15.138:6789/0 conn(0x7f0188009eb0 legacy=0x7f018800a290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.138:6789/0 2024-01-21T22:24:21.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:21.817+0000 7f01a7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f018800aa40 0x7f018800e0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-21T22:24:22.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:22.018+0000 7f01af400640 1 -- >> v1:172.21.15.138:6789/0 conn(0x7f0188009eb0 legacy=0x7f018800a290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.138:6789/0 2024-01-21T22:24:22.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:22.018+0000 7f01a7e00640 1 -- >> v2:172.21.15.152:3300/0 conn(0x7f018800aa40 msgr2=0x7f018800e0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.152:3300/0 2024-01-21T22:24:22.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:22.018+0000 7f01afe00640 1 -- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0188008d10 msgr2=0x7f01880085e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.152:3301/0 2024-01-21T22:24:22.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:22.018+0000 7f01a7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f018800aa40 0x7f018800e0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-21T22:24:22.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:22.018+0000 7f01afe00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0188008d10 0x7f01880085e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-21T22:24:22.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:22.419+0000 7f01af400640 1 -- >> v1:172.21.15.138:6789/0 conn(0x7f0188009eb0 legacy=0x7f018800a290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.138:6789/0 2024-01-21T22:24:22.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:22.419+0000 7f01a7e00640 1 -- >> v2:172.21.15.152:3300/0 conn(0x7f018800aa40 msgr2=0x7f018800e0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.152:3300/0 2024-01-21T22:24:22.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:22.419+0000 7f01afe00640 1 -- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0188008d10 msgr2=0x7f01880085e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.152:3301/0 2024-01-21T22:24:22.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:22.419+0000 7f01a7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f018800aa40 0x7f018800e0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-21T22:24:22.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:22.419+0000 7f01afe00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0188008d10 0x7f01880085e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-21T22:24:22.864 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:00:07.121 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:24:22.865 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:07.122 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:24:23.183 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:00:07.440 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:24:23.184 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:00:07.442 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:24:23.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:23.220+0000 7f01af400640 1 -- >> v1:172.21.15.138:6789/0 conn(0x7f0188009eb0 legacy=0x7f018800a290 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.138:6789/0 2024-01-21T22:24:23.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:23.220+0000 7f01a7e00640 1 -- >> v2:172.21.15.152:3300/0 conn(0x7f018800aa40 msgr2=0x7f018800e0b0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.152:3300/0 2024-01-21T22:24:23.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:23.220+0000 7f01afe00640 1 -- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0188008d10 msgr2=0x7f01880085e0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v2:172.21.15.152:3301/0 2024-01-21T22:24:23.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:23.220+0000 7f01a7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f018800aa40 0x7f018800e0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-21T22:24:23.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:23.220+0000 7f01afe00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0188008d10 0x7f01880085e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-21T22:24:23.257 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:00:07.450 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:24:23.577 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:00:07.769 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:24:24.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:24.817+0000 7f01ad600640 1 -- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0188008d10 msgr2=0x7f01880085e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-21T22:24:24.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:24.817+0000 7f01ad600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0188008d10 0x7f01880085e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:24.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:24.817+0000 7f01ad600640 1 -- >> v2:172.21.15.152:3300/0 conn(0x7f018800aa40 msgr2=0x7f018800e0b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-21T22:24:24.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:24.817+0000 7f01ad600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f018800aa40 0x7f018800e0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:24.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:24.817+0000 7f01ad600640 1 -- >> v1:172.21.15.138:6789/0 conn(0x7f0188009eb0 legacy=0x7f018800a290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-21T22:24:24.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:24.817+0000 7f01ad600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f0188011610 0x7f0188013a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:24.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:24.817+0000 7f01ad600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b00ca110 con 0x7f01880091a0 2024-01-21T22:24:24.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:24.817+0000 7f01ad600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b015a570 con 0x7f0188011610 2024-01-21T22:24:24.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:24.875+0000 7f01af400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f01880091a0 0x7f01880095b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:38518/0 (socket says 172.21.15.138:38518) 2024-01-21T22:24:24.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:24.875+0000 7f01af400640 1 -- 172.21.15.138:0/817185874 learned_addr learned my addr 172.21.15.138:0/817185874 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:24:25.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:25.167+0000 7f01a7e00640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.138:3300/0 conn(0x7f0188011610 0x7f0188013a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:25.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:25.565+0000 7f01a7e00640 1 -- 172.21.15.138:0/817185874 >> v2:172.21.15.138:3300/0 conn(0x7f0188011610 msgr2=0x7f0188013a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-01-21T22:24:25.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:25.565+0000 7f01a7e00640 1 -- 172.21.15.138:0/817185874 >> v2:172.21.15.138:3300/0 conn(0x7f0188011610 msgr2=0x7f0188013a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-21T22:24:25.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:25.565+0000 7f01a7e00640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.138:3300/0 conn(0x7f0188011610 0x7f0188013a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-21T22:24:25.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:25.565+0000 7f01a7e00640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.138:3300/0 conn(0x7f0188011610 0x7f0188013a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-21T22:24:26.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:26.118+0000 7f01a7e00640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.138:3300/0 conn(0x7f0188011610 0x7f0188013a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:26.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:26.143+0000 7f01a7e00640 1 -- 172.21.15.138:0/817185874 >> v2:172.21.15.138:3300/0 conn(0x7f0188011610 msgr2=0x7f0188013a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-01-21T22:24:26.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:26.143+0000 7f01a7e00640 1 -- 172.21.15.138:0/817185874 >> v2:172.21.15.138:3300/0 conn(0x7f0188011610 msgr2=0x7f0188013a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-21T22:24:26.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:26.143+0000 7f01a7e00640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.138:3300/0 conn(0x7f0188011610 0x7f0188013a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-21T22:24:26.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:26.143+0000 7f01a7e00640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.138:3300/0 conn(0x7f0188011610 0x7f0188013a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-21T22:24:26.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:26.595+0000 7f01a7e00640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.138:3300/0 conn(0x7f0188011610 0x7f0188013a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:26.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:26.612+0000 7f01a7e00640 1 -- 172.21.15.138:0/817185874 >> v2:172.21.15.138:3300/0 conn(0x7f0188011610 msgr2=0x7f0188013a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-01-21T22:24:26.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:26.612+0000 7f01a7e00640 1 -- 172.21.15.138:0/817185874 >> v2:172.21.15.138:3300/0 conn(0x7f0188011610 msgr2=0x7f0188013a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-21T22:24:26.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:26.612+0000 7f01a7e00640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.138:3300/0 conn(0x7f0188011610 0x7f0188013a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-21T22:24:26.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:26.612+0000 7f01a7e00640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.138:3300/0 conn(0x7f0188011610 0x7f0188013a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-21T22:24:27.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:27.817+0000 7f01ad600640 1 -- 172.21.15.138:0/817185874 >> v2:172.21.15.138:3300/0 conn(0x7f0188011610 msgr2=0x7f0188013a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:27.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:27.817+0000 7f01ad600640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.138:3300/0 conn(0x7f0188011610 0x7f0188013a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:27.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:27.818+0000 7f01ad600640 1 -- 172.21.15.138:0/817185874 >> v1:172.21.15.152:6789/0 conn(0x7f01880091a0 legacy=0x7f01880095b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:27.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:27.818+0000 7f01afe00640 1 -- 172.21.15.138:0/817185874 reap_dead start 2024-01-21T22:24:27.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:27.818+0000 7f01ad600640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.152:3300/0 conn(0x7f018800a290 0x7f01880140f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:27.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:27.818+0000 7f01ad600640 1 --2- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 0x7f0188013a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:27.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:27.818+0000 7f01ad600640 1 -- 172.21.15.138:0/817185874 --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b00c7290 con 0x7f0188014630 2024-01-21T22:24:27.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:27.818+0000 7f01ad600640 1 -- 172.21.15.138:0/817185874 --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b015a570 con 0x7f018800a290 2024-01-21T22:24:27.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:27.818+0000 7f01ad600640 1 -- 172.21.15.138:0/817185874 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b00cb640 con 0x7f01880085e0 2024-01-21T22:24:27.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:27.820+0000 7f01af400640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.152:3300/0 conn(0x7f018800a290 0x7f01880140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:27.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:27.821+0000 7f01af400640 1 -- 172.21.15.138:0/817185874 >> v2:172.21.15.152:3300/0 conn(0x7f018800a290 msgr2=0x7f01880140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-01-21T22:24:27.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:27.821+0000 7f01af400640 1 -- 172.21.15.138:0/817185874 >> v2:172.21.15.152:3300/0 conn(0x7f018800a290 msgr2=0x7f01880140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-21T22:24:27.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:27.821+0000 7f01af400640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.152:3300/0 conn(0x7f018800a290 0x7f01880140f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-21T22:24:27.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:27.821+0000 7f01af400640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.152:3300/0 conn(0x7f018800a290 0x7f01880140f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-21T22:24:27.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:27.841+0000 7f01afe00640 1 --2- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 0x7f0188013a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:27.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:27.842+0000 7f01afe00640 1 -- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 msgr2=0x7f0188013a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-21T22:24:27.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:27.842+0000 7f01afe00640 1 -- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 msgr2=0x7f0188013a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-21T22:24:27.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:27.842+0000 7f01afe00640 1 --2- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 0x7f0188013a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-21T22:24:27.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:27.842+0000 7f01afe00640 1 --2- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 0x7f0188013a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-01-21T22:24:28.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:28.050+0000 7f01af400640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.152:3300/0 conn(0x7f018800a290 0x7f01880140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:28.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:28.058+0000 7f01af400640 1 -- 172.21.15.138:0/817185874 >> v2:172.21.15.152:3300/0 conn(0x7f018800a290 msgr2=0x7f01880140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-21T22:24:28.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:28.059+0000 7f01af400640 1 -- 172.21.15.138:0/817185874 >> v2:172.21.15.152:3300/0 conn(0x7f018800a290 msgr2=0x7f01880140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-21T22:24:28.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:28.059+0000 7f01af400640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.152:3300/0 conn(0x7f018800a290 0x7f01880140f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-21T22:24:28.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:28.059+0000 7f01af400640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.152:3300/0 conn(0x7f018800a290 0x7f01880140f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-21T22:24:28.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:28.060+0000 7f01afe00640 1 --2- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 0x7f0188013a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:28.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:28.062+0000 7f01afe00640 1 -- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 msgr2=0x7f0188013a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-01-21T22:24:28.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:28.062+0000 7f01afe00640 1 -- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 msgr2=0x7f0188013a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-21T22:24:28.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:28.062+0000 7f01afe00640 1 --2- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 0x7f0188013a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-21T22:24:28.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:28.062+0000 7f01afe00640 1 --2- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 0x7f0188013a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-01-21T22:24:28.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:28.463+0000 7f01afe00640 1 --2- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 0x7f0188013a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:28.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:28.463+0000 7f01af400640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.152:3300/0 conn(0x7f018800a290 0x7f01880140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:28.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:28.465+0000 7f01afe00640 1 -- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 msgr2=0x7f0188013a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-01-21T22:24:28.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:28.465+0000 7f01afe00640 1 -- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 msgr2=0x7f0188013a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-21T22:24:28.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:28.465+0000 7f01afe00640 1 --2- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 0x7f0188013a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-21T22:24:28.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:28.465+0000 7f01afe00640 1 --2- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 0x7f0188013a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-21T22:24:28.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:28.473+0000 7f01af400640 1 -- 172.21.15.138:0/817185874 >> v2:172.21.15.152:3300/0 conn(0x7f018800a290 msgr2=0x7f01880140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-21T22:24:28.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:28.473+0000 7f01af400640 1 -- 172.21.15.138:0/817185874 >> v2:172.21.15.152:3300/0 conn(0x7f018800a290 msgr2=0x7f01880140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-21T22:24:28.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:28.473+0000 7f01af400640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.152:3300/0 conn(0x7f018800a290 0x7f01880140f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-21T22:24:28.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:28.473+0000 7f01af400640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.152:3300/0 conn(0x7f018800a290 0x7f01880140f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-01-21T22:24:29.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:29.274+0000 7f01afe00640 1 --2- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 0x7f0188013a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:29.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:29.425+0000 7f01af400640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.152:3300/0 conn(0x7f018800a290 0x7f01880140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:29.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:29.431+0000 7f01afe00640 1 -- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 msgr2=0x7f0188013a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-01-21T22:24:29.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:29.431+0000 7f01afe00640 1 -- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 msgr2=0x7f0188013a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-21T22:24:29.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:29.431+0000 7f01afe00640 1 --2- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 0x7f0188013a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-21T22:24:29.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:29.431+0000 7f01afe00640 1 --2- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 0x7f0188013a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-21T22:24:29.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:29.432+0000 7f01af400640 1 -- 172.21.15.138:0/817185874 >> v2:172.21.15.152:3300/0 conn(0x7f018800a290 msgr2=0x7f01880140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-01-21T22:24:29.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:29.432+0000 7f01af400640 1 -- 172.21.15.138:0/817185874 >> v2:172.21.15.152:3300/0 conn(0x7f018800a290 msgr2=0x7f01880140f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-01-21T22:24:29.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:29.432+0000 7f01af400640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.152:3300/0 conn(0x7f018800a290 0x7f01880140f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-01-21T22:24:29.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:29.432+0000 7f01af400640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.152:3300/0 conn(0x7f018800a290 0x7f01880140f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-01-21T22:24:29.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:29.920+0000 7f01aea00640 1 -- 172.21.15.138:0/817185874 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481596995 0 0) 0x7f01b00c7290 con 0x7f0188014630 2024-01-21T22:24:29.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:29.921+0000 7f01aea00640 1 -- 172.21.15.138:0/817185874 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0190003520 con 0x7f0188014630 2024-01-21T22:24:30.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.347+0000 7f01aea00640 1 -- 172.21.15.138:0/817185874 <== mon.1 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1084892719 0 0) 0x7f0190003520 con 0x7f0188014630 2024-01-21T22:24:30.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.347+0000 7f01aea00640 1 -- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 msgr2=0x7f0188013a00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-21T22:24:30.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.347+0000 7f01aea00640 1 --2- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 0x7f0188013a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:30.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.347+0000 7f01aea00640 1 -- 172.21.15.138:0/817185874 >> v2:172.21.15.152:3300/0 conn(0x7f018800a290 msgr2=0x7f01880140f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-01-21T22:24:30.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.347+0000 7f01aea00640 1 --2- 172.21.15.138:0/817185874 >> v2:172.21.15.152:3300/0 conn(0x7f018800a290 0x7f01880140f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:30.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.347+0000 7f01aea00640 1 -- 172.21.15.138:0/817185874 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0188014a90 con 0x7f0188014630 2024-01-21T22:24:30.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.348+0000 7f01aea00640 1 -- 172.21.15.138:0/817185874 <== mon.1 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1288616872 0 0) 0x7f0198009430 con 0x7f0188014630 2024-01-21T22:24:30.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.348+0000 7f01aea00640 1 -- 172.21.15.138:0/817185874 >> v1:172.21.15.152:6789/0 conn(0x7f0188014630 legacy=0x7f018800b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:30.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.348+0000 7f01aea00640 1 --2- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0190003970 0x7f0190003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:30.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.348+0000 7f01aea00640 1 --2- 172.21.15.138:0/817185874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01900047c0 0x7f0190004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:30.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.348+0000 7f01aea00640 1 --2- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 0x7f0190005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:30.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.348+0000 7f01aea00640 1 -- 172.21.15.138:0/817185874 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b015a570 con 0x7f01900047c0 2024-01-21T22:24:30.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.348+0000 7f01aea00640 1 -- 172.21.15.138:0/817185874 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b00cb640 con 0x7f0190003970 2024-01-21T22:24:30.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.348+0000 7f01aea00640 1 -- 172.21.15.138:0/817185874 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019000ba80 con 0x7f01880085e0 2024-01-21T22:24:30.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.364+0000 7f01af400640 1 --2- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0190003970 0x7f0190003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:30.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.382+0000 7f01af400640 1 -- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 msgr2=0x7f0190005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:30.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.382+0000 7f01af400640 1 --2- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01880085e0 0x7f0190005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:30.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.382+0000 7f01af400640 1 -- 172.21.15.138:0/817185874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01900047c0 msgr2=0x7f0190004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:30.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.382+0000 7f01af400640 1 --2- 172.21.15.138:0/817185874 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01900047c0 0x7f0190004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:30.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.382+0000 7f01af400640 1 -- 172.21.15.138:0/817185874 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0188014a90 con 0x7f0190003970 2024-01-21T22:24:30.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.387+0000 7f01af400640 1 --2- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0190003970 0x7f0190003d90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0188009cc0 tx=0x7f01900057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32302631a5771596 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:30.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.393+0000 7f01aea00640 1 -- 172.21.15.138:0/817185874 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a002d8d0 con 0x7f0190003970 2024-01-21T22:24:30.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.393+0000 7f01aea00640 1 -- 172.21.15.138:0/817185874 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a0031070 con 0x7f0190003970 2024-01-21T22:24:30.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.393+0000 7f01aea00640 1 -- 172.21.15.138:0/817185874 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a002dc90 con 0x7f0190003970 2024-01-21T22:24:30.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.395+0000 7f01b6600640 1 -- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0190003970 msgr2=0x7f0190003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:30.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.395+0000 7f01b6600640 1 --2- 172.21.15.138:0/817185874 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0190003970 0x7f0190003d90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0188009cc0 tx=0x7f01900057f0 comp rx=0 tx=0).stop 2024-01-21T22:24:30.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.395+0000 7f01afe00640 1 -- 172.21.15.138:0/817185874 reap_dead start 2024-01-21T22:24:30.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.395+0000 7f01b6600640 1 -- 172.21.15.138:0/817185874 shutdown_connections 2024-01-21T22:24:30.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.395+0000 7f01b6600640 1 -- 172.21.15.138:0/817185874 >> 172.21.15.138:0/817185874 conn(0x7f01b00cf430 msgr2=0x7f01b00cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:30.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.395+0000 7f01b6600640 1 -- 172.21.15.138:0/817185874 shutdown_connections 2024-01-21T22:24:30.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.395+0000 7f01b6600640 1 -- 172.21.15.138:0/817185874 wait complete. 2024-01-21T22:24:30.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.400+0000 7f01b6600640 1 Processor -- start 2024-01-21T22:24:30.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.400+0000 7f01b6600640 1 -- start start 2024-01-21T22:24:30.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.402+0000 7f01b6600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01b01f38a0 0x7f01b01f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:30.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.402+0000 7f01b6600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01b01532d0 0x7f01b01563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:30.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.402+0000 7f01b6600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01b01568e0 0x7f01b00ce240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:30.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.402+0000 7f01b6600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b01f3470 con 0x7f01b01532d0 2024-01-21T22:24:30.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.402+0000 7f01b6600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b01efc80 con 0x7f01b01f38a0 2024-01-21T22:24:30.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.402+0000 7f01b6600640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b01eff40 con 0x7f01b01568e0 2024-01-21T22:24:30.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.403+0000 7f01afe00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01b01568e0 0x7f01b00ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:30.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.403+0000 7f01af400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01b01f38a0 0x7f01b01f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:30.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.403+0000 7f01afe00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01b01568e0 0x7f01b00ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:59956/0 (socket says 172.21.15.138:59956) 2024-01-21T22:24:30.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.403+0000 7f01afe00640 1 -- 172.21.15.138:0/2899497766 learned_addr learned my addr 172.21.15.138:0/2899497766 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:24:30.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.407+0000 7f01afe00640 1 -- 172.21.15.138:0/2899497766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01b01f38a0 msgr2=0x7f01b01f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:30.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.407+0000 7f01afe00640 1 --2- 172.21.15.138:0/2899497766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01b01f38a0 0x7f01b01f3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:30.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.407+0000 7f01afe00640 1 -- 172.21.15.138:0/2899497766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01b01532d0 msgr2=0x7f01b01563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:30.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.407+0000 7f01afe00640 1 --2- 172.21.15.138:0/2899497766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01b01532d0 0x7f01b01563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:30.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.407+0000 7f01afe00640 1 -- 172.21.15.138:0/2899497766 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b01541c0 con 0x7f01b01568e0 2024-01-21T22:24:30.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.409+0000 7f01afe00640 1 --2- 172.21.15.138:0/2899497766 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01b01568e0 0x7f01b00ce240 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f01b00c6890 tx=0x7f01a800ab10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:30.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.413+0000 7f01a7400640 1 -- 172.21.15.138:0/2899497766 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a80106c0 con 0x7f01b01568e0 2024-01-21T22:24:30.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.413+0000 7f01a7400640 1 -- 172.21.15.138:0/2899497766 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a800d040 con 0x7f01b01568e0 2024-01-21T22:24:30.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.413+0000 7f01a7400640 1 -- 172.21.15.138:0/2899497766 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a8010a80 con 0x7f01b01568e0 2024-01-21T22:24:30.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.413+0000 7f01b6600640 1 -- 172.21.15.138:0/2899497766 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b01f0220 con 0x7f01b01568e0 2024-01-21T22:24:30.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.414+0000 7f01b6600640 1 -- 172.21.15.138:0/2899497766 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b01f07e0 con 0x7f01b01568e0 2024-01-21T22:24:30.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.415+0000 7f01b6600640 1 -- 172.21.15.138:0/2899497766 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b01519e0 con 0x7f01b01568e0 2024-01-21T22:24:30.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.610+0000 7f01a7400640 1 -- 172.21.15.138:0/2899497766 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7f01a801e460 con 0x7f01b01568e0 2024-01-21T22:24:30.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.618+0000 7f01a7400640 1 -- 172.21.15.138:0/2899497766 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f01a8028730 con 0x7f01b01568e0 2024-01-21T22:24:30.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.619+0000 7f01a7400640 1 -- 172.21.15.138:0/2899497766 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7f01b01519e0 con 0x7f01b01568e0 2024-01-21T22:24:30.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:30.699+0000 7f01b6600640 1 -- 172.21.15.138:0/2899497766 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f01b00c88e0 con 0x7f01b01568e0 2024-01-21T22:24:30.833 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:30.832+0000 7fc104e38040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-21T22:24:30.885 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:30.884+0000 7fc104e38040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-21T22:24:30.986 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:30.984+0000 7f8d15c41040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-01-21T22:24:31.071 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:31.069+0000 7f8d15c41040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-01-21T22:24:31.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:31.232+0000 7f01a7400640 1 -- 172.21.15.138:0/2899497766 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f01a802c090 con 0x7f01b01568e0 2024-01-21T22:24:31.234 INFO:teuthology.orchestra.run.smithi138.stderr:adjusted tunables profile to default 2024-01-21T22:24:31.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:31.234+0000 7f01b6600640 1 -- 172.21.15.138:0/2899497766 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01b01568e0 msgr2=0x7f01b00ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:31.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:31.234+0000 7f01b6600640 1 --2- 172.21.15.138:0/2899497766 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01b01568e0 0x7f01b00ce240 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f01b00c6890 tx=0x7f01a800ab10 comp rx=0 tx=0).stop 2024-01-21T22:24:31.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:31.234+0000 7f01b6600640 1 -- 172.21.15.138:0/2899497766 shutdown_connections 2024-01-21T22:24:31.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:31.234+0000 7f01b6600640 1 --2- 172.21.15.138:0/2899497766 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01b01568e0 0x7f01b00ce240 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:31.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:31.234+0000 7f01b6600640 1 --2- 172.21.15.138:0/2899497766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01b01532d0 0x7f01b01563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:31.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:31.234+0000 7f01b6600640 1 --2- 172.21.15.138:0/2899497766 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01b01f38a0 0x7f01b01f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:31.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:31.234+0000 7f01b6600640 1 -- 172.21.15.138:0/2899497766 >> 172.21.15.138:0/2899497766 conn(0x7f01b00cf430 msgr2=0x7f01b014e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:31.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:31.234+0000 7f01b6600640 1 -- 172.21.15.138:0/2899497766 shutdown_connections 2024-01-21T22:24:31.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:31.234+0000 7f01b6600640 1 -- 172.21.15.138:0/2899497766 wait complete. 2024-01-21T22:24:31.261 INFO:tasks.ceph:check_enable_crimson: False 2024-01-21T22:24:31.261 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-01-21T22:24:31.261 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:24:31.261 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-01-21T22:24:31.345 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:24:31.345 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-01-21T22:24:31.402 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:31.401+0000 7fc104e38040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-21T22:24:31.458 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:31.457+0000 7fc104e38040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-21T22:24:31.472 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:24:31.472 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-01-21T22:24:31.504 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:31.503+0000 7fc104e38040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-21T22:24:31.552 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:24:31.552 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-01-21T22:24:31.558 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:31.557+0000 7fc104e38040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-21T22:24:31.600 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:31.598+0000 7fc104e38040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-21T22:24:31.667 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-21T22:24:31.667 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-01-21T22:24:31.694 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-21T22:24:31.695 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-01-21T22:24:31.704 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:31.703+0000 7fc104e38040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-21T22:24:31.730 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-21T22:24:31.730 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-01-21T22:24:31.746 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:31.744+0000 7f8d15c41040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-01-21T22:24:31.776 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:31.775+0000 7fc104e38040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-21T22:24:31.796 DEBUG:teuthology.orchestra.run.smithi152:> set -ex 2024-01-21T22:24:31.797 DEBUG:teuthology.orchestra.run.smithi152:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-01-21T22:24:31.808 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:31.805+0000 7f8d15c41040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-01-21T22:24:31.824 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:31.823+0000 7fc104e38040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-21T22:24:31.850 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new d812b2d6-c8a3-4015-bfdb-5766b25eae5a 0 2024-01-21T22:24:31.860 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:31.858+0000 7f8d15c41040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-01-21T22:24:31.879 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:31.877+0000 7fc104e38040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-21T22:24:31.925 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:31.923+0000 7f8d15c41040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-01-21T22:24:31.928 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:31.927+0000 7fc104e38040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-21T22:24:31.978 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:31.976+0000 7f8d15c41040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-01-21T22:24:31.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.977+0000 7fdc9b91d640 1 Processor -- start 2024-01-21T22:24:31.979 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.978+0000 7fdc9b91d640 1 -- start start 2024-01-21T22:24:31.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.978+0000 7fdc9b91d640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdc940c93d0 0x7fdc940718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:31.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.978+0000 7fdc9b91d640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdc940712d0 0x7fdc94071e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:31.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.978+0000 7fdc9b91d640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdc940c97b0 0x7fdc9415a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:31.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.978+0000 7fdc9b91d640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc940c7100 con 0x7fdc940c93d0 2024-01-21T22:24:31.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.978+0000 7fdc9b91d640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc940c73c0 con 0x7fdc940712d0 2024-01-21T22:24:31.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.978+0000 7fdc9b91d640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc940c76a0 con 0x7fdc940c97b0 2024-01-21T22:24:31.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.979+0000 7fdc99eba640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdc940c97b0 0x7fdc9415a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:31.980 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.979+0000 7fdc98eb8640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdc940712d0 0x7fdc94071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:31.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.979+0000 7fdc99eba640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdc940c97b0 0x7fdc9415a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.152:36522/0 (socket says 172.21.15.152:36522) 2024-01-21T22:24:31.981 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.979+0000 7fdc99eba640 1 -- 172.21.15.152:0/2614568522 learned_addr learned my addr 172.21.15.152:0/2614568522 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-01-21T22:24:31.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.982+0000 7fdc99eba640 1 -- 172.21.15.152:0/2614568522 >> v2:172.21.15.152:3300/0 conn(0x7fdc940712d0 msgr2=0x7fdc94071e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:31.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.982+0000 7fdc99eba640 1 --2- 172.21.15.152:0/2614568522 >> v2:172.21.15.152:3300/0 conn(0x7fdc940712d0 0x7fdc94071e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:31.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.982+0000 7fdc99eba640 1 -- 172.21.15.152:0/2614568522 >> v2:172.21.15.138:3300/0 conn(0x7fdc940c93d0 msgr2=0x7fdc940718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:31.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.982+0000 7fdc99eba640 1 --2- 172.21.15.152:0/2614568522 >> v2:172.21.15.138:3300/0 conn(0x7fdc940c93d0 0x7fdc940718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:31.983 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.982+0000 7fdc99eba640 1 -- 172.21.15.152:0/2614568522 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc940c7980 con 0x7fdc940c97b0 2024-01-21T22:24:31.985 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.984+0000 7fdc99eba640 1 --2- 172.21.15.152:0/2614568522 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdc940c97b0 0x7fdc9415a430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdc9406c830 tx=0x7fdc8c02e170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd765cb4d5aa3ada server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:31.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.987+0000 7fdc8bfff640 1 -- 172.21.15.152:0/2614568522 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c00b0d0 con 0x7fdc940c97b0 2024-01-21T22:24:31.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.987+0000 7fdc8bfff640 1 -- 172.21.15.152:0/2614568522 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8c00b230 con 0x7fdc940c97b0 2024-01-21T22:24:31.988 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.987+0000 7fdc8bfff640 1 -- 172.21.15.152:0/2614568522 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c009600 con 0x7fdc940c97b0 2024-01-21T22:24:31.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.988+0000 7fdc9b91d640 1 -- 172.21.15.152:0/2614568522 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdc940c97b0 msgr2=0x7fdc9415a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:31.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.988+0000 7fdc9b91d640 1 --2- 172.21.15.152:0/2614568522 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdc940c97b0 0x7fdc9415a430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdc9406c830 tx=0x7fdc8c02e170 comp rx=0 tx=0).stop 2024-01-21T22:24:31.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.988+0000 7fdc9b91d640 1 -- 172.21.15.152:0/2614568522 shutdown_connections 2024-01-21T22:24:31.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.988+0000 7fdc9b91d640 1 --2- 172.21.15.152:0/2614568522 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdc940c97b0 0x7fdc9415a430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:31.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.988+0000 7fdc9b91d640 1 --2- 172.21.15.152:0/2614568522 >> v2:172.21.15.152:3300/0 conn(0x7fdc940712d0 0x7fdc94071e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:31.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.988+0000 7fdc9b91d640 1 --2- 172.21.15.152:0/2614568522 >> v2:172.21.15.138:3300/0 conn(0x7fdc940c93d0 0x7fdc940718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:31.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.988+0000 7fdc9b91d640 1 -- 172.21.15.152:0/2614568522 >> 172.21.15.152:0/2614568522 conn(0x7fdc9406f200 msgr2=0x7fdc9406f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:31.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.988+0000 7fdc9b91d640 1 -- 172.21.15.152:0/2614568522 shutdown_connections 2024-01-21T22:24:31.990 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.988+0000 7fdc9b91d640 1 -- 172.21.15.152:0/2614568522 wait complete. 2024-01-21T22:24:31.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.990+0000 7fdc9b91d640 1 Processor -- start 2024-01-21T22:24:31.991 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.990+0000 7fdc9b91d640 1 -- start start 2024-01-21T22:24:31.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.991+0000 7fdc9b91d640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc940712d0 0x7fdc940cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:31.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.991+0000 7fdc9b91d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc940c93d0 0x7fdc940c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:31.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.991+0000 7fdc9b91d640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdc940c97b0 0x7fdc941534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:31.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.991+0000 7fdc9b91d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc941ef3a0 con 0x7fdc940c93d0 2024-01-21T22:24:31.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.991+0000 7fdc9b91d640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc940a9d20 con 0x7fdc940712d0 2024-01-21T22:24:31.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.991+0000 7fdc9b91d640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc940c6400 con 0x7fdc940c97b0 2024-01-21T22:24:31.993 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.992+0000 7fdc99eba640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdc940c97b0 0x7fdc941534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:31.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.992+0000 7fdc99eba640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdc940c97b0 0x7fdc941534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.152:36528/0 (socket says 172.21.15.152:36528) 2024-01-21T22:24:31.994 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.992+0000 7fdc99eba640 1 -- 172.21.15.152:0/2576950005 learned_addr learned my addr 172.21.15.152:0/2576950005 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-01-21T22:24:31.998 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.997+0000 7fdc996b9640 1 --2- 172.21.15.152:0/2576950005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc940712d0 0x7fdc940cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:31.999 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:31.998+0000 7fdc98eb8640 1 --2- 172.21.15.152:0/2576950005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc940c93d0 0x7fdc940c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:32.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.002+0000 7fdc99eba640 1 -- 172.21.15.152:0/2576950005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc940712d0 msgr2=0x7fdc940cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:32.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.002+0000 7fdc99eba640 1 --2- 172.21.15.152:0/2576950005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc940712d0 0x7fdc940cbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:32.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.002+0000 7fdc99eba640 1 -- 172.21.15.152:0/2576950005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc940c93d0 msgr2=0x7fdc940c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:32.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.002+0000 7fdc99eba640 1 --2- 172.21.15.152:0/2576950005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc940c93d0 0x7fdc940c5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:32.003 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.002+0000 7fdc99eba640 1 -- 172.21.15.152:0/2576950005 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c00bd40 con 0x7fdc940c97b0 2024-01-21T22:24:32.006 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.005+0000 7fdc99eba640 1 --2- 172.21.15.152:0/2576950005 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdc940c97b0 0x7fdc941534f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c038040 tx=0x7fdc8c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:32.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.008+0000 7fdc89ffb640 1 -- 172.21.15.152:0/2576950005 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c00b720 con 0x7fdc940c97b0 2024-01-21T22:24:32.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.008+0000 7fdc89ffb640 1 -- 172.21.15.152:0/2576950005 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8c009e10 con 0x7fdc940c97b0 2024-01-21T22:24:32.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.008+0000 7fdc89ffb640 1 -- 172.21.15.152:0/2576950005 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c0408a0 con 0x7fdc940c97b0 2024-01-21T22:24:32.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.008+0000 7fdc9b91d640 1 -- 172.21.15.152:0/2576950005 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc941ee790 con 0x7fdc940c97b0 2024-01-21T22:24:32.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.009+0000 7fdc9b91d640 1 -- 172.21.15.152:0/2576950005 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc941eed50 con 0x7fdc940c97b0 2024-01-21T22:24:32.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.009+0000 7fdc9b91d640 1 -- 172.21.15.152:0/2576950005 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc941ef550 con 0x7fdc940c97b0 2024-01-21T22:24:32.105 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:32.103+0000 7f8d15c41040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-01-21T22:24:32.189 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:32.186+0000 7f8d15c41040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-01-21T22:24:32.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.191+0000 7fdc89ffb640 1 -- 172.21.15.152:0/2576950005 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7fdc8c040a00 con 0x7fdc940c97b0 2024-01-21T22:24:32.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.191+0000 7fdc89ffb640 1 -- 172.21.15.152:0/2576950005 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdc8c053ca0 con 0x7fdc940c97b0 2024-01-21T22:24:32.193 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.192+0000 7fdc89ffb640 1 -- 172.21.15.152:0/2576950005 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7fdc941ef550 con 0x7fdc940c97b0 2024-01-21T22:24:32.245 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:32.242+0000 7f8d15c41040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-01-21T22:24:32.266 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.265+0000 7fdc9b91d640 1 -- 172.21.15.152:0/2576950005 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d812b2d6-c8a3-4015-bfdb-5766b25eae5a", "id": 0} v 0) v1 -- 0x7fdc9406ae00 con 0x7fdc940c97b0 2024-01-21T22:24:32.307 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:32.305+0000 7f8d15c41040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-01-21T22:24:32.365 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:32.363+0000 7f8d15c41040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-01-21T22:24:32.394 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:32.393+0000 7fc104e38040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-21T22:24:32.490 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.489+0000 7fdc89ffb640 1 -- 172.21.15.152:0/2576950005 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d812b2d6-c8a3-4015-bfdb-5766b25eae5a", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdc8c057020 con 0x7fdc940c97b0 2024-01-21T22:24:32.490 INFO:teuthology.orchestra.run.smithi152.stdout:0 2024-01-21T22:24:32.491 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.490+0000 7fdc9b91d640 1 -- 172.21.15.152:0/2576950005 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdc940c97b0 msgr2=0x7fdc941534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:32.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.490+0000 7fdc9b91d640 1 --2- 172.21.15.152:0/2576950005 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdc940c97b0 0x7fdc941534f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c038040 tx=0x7fdc8c004a40 comp rx=0 tx=0).stop 2024-01-21T22:24:32.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.490+0000 7fdc9b91d640 1 -- 172.21.15.152:0/2576950005 shutdown_connections 2024-01-21T22:24:32.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.490+0000 7fdc9b91d640 1 --2- 172.21.15.152:0/2576950005 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdc940c97b0 0x7fdc941534f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:32.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.490+0000 7fdc9b91d640 1 --2- 172.21.15.152:0/2576950005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc940c93d0 0x7fdc940c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:32.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.490+0000 7fdc9b91d640 1 --2- 172.21.15.152:0/2576950005 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc940712d0 0x7fdc940cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:32.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.490+0000 7fdc9b91d640 1 -- 172.21.15.152:0/2576950005 >> 172.21.15.152:0/2576950005 conn(0x7fdc9406f200 msgr2=0x7fdc9406a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:32.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.490+0000 7fdc9b91d640 1 -- 172.21.15.152:0/2576950005 shutdown_connections 2024-01-21T22:24:32.492 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.490+0000 7fdc9b91d640 1 -- 172.21.15.152:0/2576950005 wait complete. 2024-01-21T22:24:32.501 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new d8d875df-c840-46cc-b6c0-a847f9c93999 1 2024-01-21T22:24:32.538 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:32.537+0000 7fc104e38040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-21T22:24:32.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.602+0000 7fd195d7c640 1 Processor -- start 2024-01-21T22:24:32.603 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.602+0000 7fd195d7c640 1 -- start start 2024-01-21T22:24:32.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.602+0000 7fd195d7c640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1900c2f20 0x7fd190163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:32.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.602+0000 7fd195d7c640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd190157960 con 0x7fd1901516e0 2024-01-21T22:24:32.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.602+0000 7fd195d7c640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd190157c40 con 0x7fd1900c8820 2024-01-21T22:24:32.604 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.602+0000 7fd195d7c640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd190157770 con 0x7fd1900c2f20 2024-01-21T22:24:32.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.604+0000 7fd18f7fe640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd1901516e0 0x7fd1900cd8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.152:56740/0 (socket says 172.21.15.152:56740) 2024-01-21T22:24:32.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.604+0000 7fd18f7fe640 1 -- 172.21.15.152:0/572327260 learned_addr learned my addr 172.21.15.152:0/572327260 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-01-21T22:24:32.606 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.604+0000 7fd18ffff640 1 --2- 172.21.15.152:0/572327260 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1900c2f20 0x7fd190163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:32.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.608+0000 7fd18e7fc640 1 -- 172.21.15.152:0/572327260 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2981700867 0 0) 0x7fd190157c40 con 0x7fd1900c8820 2024-01-21T22:24:32.609 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.608+0000 7fd18e7fc640 1 -- 172.21.15.152:0/572327260 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd178003520 con 0x7fd1900c8820 2024-01-21T22:24:32.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.610+0000 7fd18ffff640 1 -- 172.21.15.152:0/572327260 >> v1:172.21.15.152:6789/0 conn(0x7fd1900c8820 legacy=0x7fd1900c2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:32.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.610+0000 7fd18ffff640 1 -- 172.21.15.152:0/572327260 >> v1:172.21.15.138:6789/0 conn(0x7fd1901516e0 legacy=0x7fd1900cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:32.611 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.610+0000 7fd18ffff640 1 -- 172.21.15.152:0/572327260 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd190157f20 con 0x7fd1900c2f20 2024-01-21T22:24:32.615 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.614+0000 7fd18ffff640 1 --2- 172.21.15.152:0/572327260 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1900c2f20 0x7fd190163100 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd188009dc0 tx=0x7fd18802bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7f80599b892887a server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:32.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.618+0000 7fd18e7fc640 1 -- 172.21.15.152:0/572327260 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18802cd50 con 0x7fd1900c2f20 2024-01-21T22:24:32.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.618+0000 7fd18e7fc640 1 -- 172.21.15.152:0/572327260 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd188034440 con 0x7fd1900c2f20 2024-01-21T22:24:32.620 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.618+0000 7fd18e7fc640 1 -- 172.21.15.152:0/572327260 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1880347d0 con 0x7fd1900c2f20 2024-01-21T22:24:32.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.619+0000 7fd195d7c640 1 -- 172.21.15.152:0/572327260 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1900c2f20 msgr2=0x7fd190163100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:32.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.619+0000 7fd195d7c640 1 --2- 172.21.15.152:0/572327260 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1900c2f20 0x7fd190163100 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd188009dc0 tx=0x7fd18802bfa0 comp rx=0 tx=0).stop 2024-01-21T22:24:32.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.619+0000 7fd195d7c640 1 -- 172.21.15.152:0/572327260 shutdown_connections 2024-01-21T22:24:32.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.619+0000 7fd195d7c640 1 --2- 172.21.15.152:0/572327260 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1900c2f20 0x7fd190163100 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:32.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.619+0000 7fd195d7c640 1 -- 172.21.15.152:0/572327260 >> 172.21.15.152:0/572327260 conn(0x7fd1900703c0 msgr2=0x7fd1900707d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:32.621 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.619+0000 7fd195d7c640 1 -- 172.21.15.152:0/572327260 shutdown_connections 2024-01-21T22:24:32.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.619+0000 7fd195d7c640 1 -- 172.21.15.152:0/572327260 wait complete. 2024-01-21T22:24:32.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.621+0000 7fd195d7c640 1 Processor -- start 2024-01-21T22:24:32.622 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.621+0000 7fd195d7c640 1 -- start start 2024-01-21T22:24:32.623 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.621+0000 7fd195d7c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1900c2f20 0x7fd1900cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:32.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.621+0000 7fd195d7c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1900c8820 0x7fd1900b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:32.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.621+0000 7fd195d7c640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1901516e0 0x7fd190144d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:32.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.621+0000 7fd195d7c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1901507a0 con 0x7fd1900c8820 2024-01-21T22:24:32.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.621+0000 7fd195d7c640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19014acf0 con 0x7fd1900c2f20 2024-01-21T22:24:32.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.621+0000 7fd195d7c640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19014cea0 con 0x7fd1901516e0 2024-01-21T22:24:32.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.622+0000 7fd18ffff640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1901516e0 0x7fd190144d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:32.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.622+0000 7fd18f7fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1900c2f20 0x7fd1900cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:32.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.622+0000 7fd18ffff640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1901516e0 0x7fd190144d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.152:36536/0 (socket says 172.21.15.152:36536) 2024-01-21T22:24:32.624 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.622+0000 7fd18ffff640 1 -- 172.21.15.152:0/3919892645 learned_addr learned my addr 172.21.15.152:0/3919892645 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-01-21T22:24:32.625 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.623+0000 7fd18effd640 1 --2- 172.21.15.152:0/3919892645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1900c8820 0x7fd1900b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:32.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.626+0000 7fd18f7fe640 1 -- 172.21.15.152:0/3919892645 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1901516e0 msgr2=0x7fd190144d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:32.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.626+0000 7fd18f7fe640 1 --2- 172.21.15.152:0/3919892645 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1901516e0 0x7fd190144d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:32.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.626+0000 7fd18f7fe640 1 -- 172.21.15.152:0/3919892645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1900c8820 msgr2=0x7fd1900b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:32.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.626+0000 7fd18f7fe640 1 --2- 172.21.15.152:0/3919892645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1900c8820 0x7fd1900b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:32.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.626+0000 7fd18f7fe640 1 -- 172.21.15.152:0/3919892645 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd188009a70 con 0x7fd1900c2f20 2024-01-21T22:24:32.627 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.626+0000 7fd18ffff640 1 --2- 172.21.15.152:0/3919892645 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1901516e0 0x7fd190144d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-21T22:24:32.629 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.628+0000 7fd18f7fe640 1 --2- 172.21.15.152:0/3919892645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1900c2f20 0x7fd1900cc4d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd180000f60 tx=0x7fd18000c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:32.631 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.630+0000 7fd177fff640 1 -- 172.21.15.152:0/3919892645 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180005ce0 con 0x7fd1900c2f20 2024-01-21T22:24:32.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.630+0000 7fd195d7c640 1 -- 172.21.15.152:0/3919892645 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1900b23f0 con 0x7fd1900c2f20 2024-01-21T22:24:32.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.630+0000 7fd195d7c640 1 -- 172.21.15.152:0/3919892645 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19014f050 con 0x7fd1900c2f20 2024-01-21T22:24:32.632 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.631+0000 7fd195d7c640 1 -- 172.21.15.152:0/3919892645 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19014eeb0 con 0x7fd1900c2f20 2024-01-21T22:24:32.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.632+0000 7fd177fff640 1 -- 172.21.15.152:0/3919892645 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd180005e40 con 0x7fd1900c2f20 2024-01-21T22:24:32.633 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.632+0000 7fd177fff640 1 -- 172.21.15.152:0/3919892645 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd180001060 con 0x7fd1900c2f20 2024-01-21T22:24:32.674 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:32.673+0000 7fc104e38040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-21T22:24:32.720 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:32.719+0000 7fc104e38040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-21T22:24:32.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.809+0000 7fd177fff640 1 -- 172.21.15.152:0/3919892645 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7fd1800047e0 con 0x7fd1900c2f20 2024-01-21T22:24:32.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.809+0000 7fd177fff640 1 -- 172.21.15.152:0/3919892645 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fd18001c7f0 con 0x7fd1900c2f20 2024-01-21T22:24:32.811 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.809+0000 7fd177fff640 1 -- 172.21.15.152:0/3919892645 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7fd19014eeb0 con 0x7fd1900c2f20 2024-01-21T22:24:32.821 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:32.820+0000 7fc104e38040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-21T22:24:32.829 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:32.826+0000 7f8d15c41040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-01-21T22:24:32.882 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:32.881+0000 7fd195d7c640 1 -- 172.21.15.152:0/3919892645 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d8d875df-c840-46cc-b6c0-a847f9c93999", "id": 1} v 0) v1 -- 0x7fd1900b28b0 con 0x7fd1900c2f20 2024-01-21T22:24:32.918 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:32.917+0000 7fc104e38040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-21T22:24:32.994 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:32.992+0000 7f8d15c41040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-01-21T22:24:33.054 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.053+0000 7fd177fff640 1 -- 172.21.15.152:0/3919892645 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d8d875df-c840-46cc-b6c0-a847f9c93999", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd180021060 con 0x7fd1900c2f20 2024-01-21T22:24:33.054 INFO:teuthology.orchestra.run.smithi152.stdout:1 2024-01-21T22:24:33.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.054+0000 7fd195d7c640 1 -- 172.21.15.152:0/3919892645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1900c2f20 msgr2=0x7fd1900cc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:33.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.054+0000 7fd195d7c640 1 --2- 172.21.15.152:0/3919892645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1900c2f20 0x7fd1900cc4d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd180000f60 tx=0x7fd18000c760 comp rx=0 tx=0).stop 2024-01-21T22:24:33.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.054+0000 7fd195d7c640 1 -- 172.21.15.152:0/3919892645 shutdown_connections 2024-01-21T22:24:33.055 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.054+0000 7fd195d7c640 1 --2- 172.21.15.152:0/3919892645 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1901516e0 0x7fd190144d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:33.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.054+0000 7fd195d7c640 1 --2- 172.21.15.152:0/3919892645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1900c8820 0x7fd1900b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:33.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.054+0000 7fd195d7c640 1 --2- 172.21.15.152:0/3919892645 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1900c2f20 0x7fd1900cc4d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:33.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.054+0000 7fd195d7c640 1 -- 172.21.15.152:0/3919892645 >> 172.21.15.152:0/3919892645 conn(0x7fd1900703c0 msgr2=0x7fd19006e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:33.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.054+0000 7fd195d7c640 1 -- 172.21.15.152:0/3919892645 shutdown_connections 2024-01-21T22:24:33.056 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.054+0000 7fd195d7c640 1 -- 172.21.15.152:0/3919892645 wait complete. 2024-01-21T22:24:33.064 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:33.063+0000 7fc104e38040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-21T22:24:33.065 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 4bd94373-76ca-47a6-870b-16d96d7c9e81 2 2024-01-21T22:24:33.109 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:33.108+0000 7fc104e38040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-21T22:24:33.148 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:33.146+0000 7f8d15c41040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-01-21T22:24:33.203 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:33.201+0000 7f8d15c41040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-01-21T22:24:33.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.207+0000 7fb14cea6640 1 Processor -- start 2024-01-21T22:24:33.208 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.207+0000 7fb14cea6640 1 -- start start 2024-01-21T22:24:33.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.208+0000 7fb14cea6640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb148151e70 0x7fb1480c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:33.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.208+0000 7fb14cea6640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb1480725f0 0x7fb1480c5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:33.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.208+0000 7fb14cea6640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb148152250 con 0x7fb14806c070 2024-01-21T22:24:33.209 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.208+0000 7fb14cea6640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb148157c80 con 0x7fb1480725f0 2024-01-21T22:24:33.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.208+0000 7fb14cea6640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1480c25c0 con 0x7fb148151e70 2024-01-21T22:24:33.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.209+0000 7fb14659c640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb148151e70 0x7fb1480c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:33.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.209+0000 7fb14659c640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb148151e70 0x7fb1480c4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:34908/0 (socket says 172.21.15.152:34908) 2024-01-21T22:24:33.210 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.209+0000 7fb14659c640 1 -- 172.21.15.152:0/3644940929 learned_addr learned my addr 172.21.15.152:0/3644940929 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-01-21T22:24:33.211 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.210+0000 7fb146d9d640 1 --2- 172.21.15.152:0/3644940929 >> v2:172.21.15.138:3300/0 conn(0x7fb1480725f0 0x7fb1480c5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:33.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.211+0000 7fb14659c640 1 -- 172.21.15.152:0/3644940929 >> v2:172.21.15.138:3300/0 conn(0x7fb1480725f0 msgr2=0x7fb1480c5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:33.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.211+0000 7fb14659c640 1 --2- 172.21.15.152:0/3644940929 >> v2:172.21.15.138:3300/0 conn(0x7fb1480725f0 0x7fb1480c5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:33.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.211+0000 7fb14659c640 1 -- 172.21.15.152:0/3644940929 >> v1:172.21.15.152:6789/0 conn(0x7fb14806c070 legacy=0x7fb148158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:33.212 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.211+0000 7fb14659c640 1 -- 172.21.15.152:0/3644940929 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1480c2880 con 0x7fb148151e70 2024-01-21T22:24:33.214 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.213+0000 7fb14659c640 1 --2- 172.21.15.152:0/3644940929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb148151e70 0x7fb1480c4ee0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb1480cc770 tx=0x7fb14002bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53867ef5f7d9f162 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:33.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.220+0000 7fb14559a640 1 -- 172.21.15.152:0/3644940929 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14002cb20 con 0x7fb148151e70 2024-01-21T22:24:33.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.220+0000 7fb14559a640 1 -- 172.21.15.152:0/3644940929 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14002cc80 con 0x7fb148151e70 2024-01-21T22:24:33.221 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.220+0000 7fb14559a640 1 -- 172.21.15.152:0/3644940929 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1400343f0 con 0x7fb148151e70 2024-01-21T22:24:33.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.221+0000 7fb14cea6640 1 -- 172.21.15.152:0/3644940929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb148151e70 msgr2=0x7fb1480c4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:33.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.221+0000 7fb14cea6640 1 --2- 172.21.15.152:0/3644940929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb148151e70 0x7fb1480c4ee0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb1480cc770 tx=0x7fb14002bf80 comp rx=0 tx=0).stop 2024-01-21T22:24:33.222 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.221+0000 7fb14cea6640 1 -- 172.21.15.152:0/3644940929 shutdown_connections 2024-01-21T22:24:33.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.221+0000 7fb14cea6640 1 --2- 172.21.15.152:0/3644940929 >> v2:172.21.15.138:3300/0 conn(0x7fb1480725f0 0x7fb1480c5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:33.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.221+0000 7fb14cea6640 1 --2- 172.21.15.152:0/3644940929 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb148151e70 0x7fb1480c4ee0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:33.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.221+0000 7fb14cea6640 1 -- 172.21.15.152:0/3644940929 >> 172.21.15.152:0/3644940929 conn(0x7fb1480cb7d0 msgr2=0x7fb1480cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:33.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.221+0000 7fb14cea6640 1 -- 172.21.15.152:0/3644940929 shutdown_connections 2024-01-21T22:24:33.223 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.221+0000 7fb14cea6640 1 -- 172.21.15.152:0/3644940929 wait complete. 2024-01-21T22:24:33.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.223+0000 7fb14cea6640 1 Processor -- start 2024-01-21T22:24:33.224 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.223+0000 7fb14cea6640 1 -- start start 2024-01-21T22:24:33.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.223+0000 7fb14cea6640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb14806c070 0x7fb14814ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:33.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.223+0000 7fb14cea6640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1480725f0 0x7fb1480c11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:33.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.223+0000 7fb14cea6640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb148151e70 0x7fb1480c16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:33.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.223+0000 7fb14cea6640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14814d510 con 0x7fb14806c070 2024-01-21T22:24:33.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.223+0000 7fb14cea6640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1480bfa90 con 0x7fb1480725f0 2024-01-21T22:24:33.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.223+0000 7fb14cea6640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb148145f10 con 0x7fb148151e70 2024-01-21T22:24:33.225 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.224+0000 7fb145d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1480725f0 0x7fb1480c11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:33.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.224+0000 7fb146d9d640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb148151e70 0x7fb1480c16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:33.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.224+0000 7fb145d9b640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1480725f0 0x7fb1480c11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:34922/0 (socket says 172.21.15.152:34922) 2024-01-21T22:24:33.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.224+0000 7fb145d9b640 1 -- 172.21.15.152:0/3805540234 learned_addr learned my addr 172.21.15.152:0/3805540234 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-01-21T22:24:33.226 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.225+0000 7fb14659c640 1 --2- 172.21.15.152:0/3805540234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb14806c070 0x7fb14814ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:33.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.228+0000 7fb145d9b640 1 -- 172.21.15.152:0/3805540234 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb148151e70 msgr2=0x7fb1480c16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:33.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.228+0000 7fb145d9b640 1 --2- 172.21.15.152:0/3805540234 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb148151e70 0x7fb1480c16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:33.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.228+0000 7fb145d9b640 1 -- 172.21.15.152:0/3805540234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb14806c070 msgr2=0x7fb14814ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:33.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.228+0000 7fb145d9b640 1 --2- 172.21.15.152:0/3805540234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb14806c070 0x7fb14814ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:33.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.228+0000 7fb145d9b640 1 -- 172.21.15.152:0/3805540234 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb140009b30 con 0x7fb1480725f0 2024-01-21T22:24:33.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.230+0000 7fb145d9b640 1 --2- 172.21.15.152:0/3805540234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1480725f0 0x7fb1480c11b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb13000b3e0 tx=0x7fb13000b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:33.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.232+0000 7fb13effd640 1 -- 172.21.15.152:0/3805540234 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb130011830 con 0x7fb1480725f0 2024-01-21T22:24:33.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.233+0000 7fb14cea6640 1 -- 172.21.15.152:0/3805540234 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1480cb5c0 con 0x7fb1480725f0 2024-01-21T22:24:33.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.233+0000 7fb14cea6640 1 -- 172.21.15.152:0/3805540234 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1481455e0 con 0x7fb1480725f0 2024-01-21T22:24:33.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.233+0000 7fb14cea6640 1 -- 172.21.15.152:0/3805540234 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb148146050 con 0x7fb1480725f0 2024-01-21T22:24:33.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.235+0000 7fb13effd640 1 -- 172.21.15.152:0/3805540234 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1300091f0 con 0x7fb1480725f0 2024-01-21T22:24:33.236 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.235+0000 7fb13effd640 1 -- 172.21.15.152:0/3805540234 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb130011b90 con 0x7fb1480725f0 2024-01-21T22:24:33.266 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:33.265+0000 7fc104e38040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-21T22:24:33.313 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:33.311+0000 7f8d15c41040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-01-21T22:24:33.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.343+0000 7fb13effd640 1 -- 172.21.15.152:0/3805540234 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7fb130024440 con 0x7fb1480725f0 2024-01-21T22:24:33.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.343+0000 7fb13effd640 1 -- 172.21.15.152:0/3805540234 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fb13001aba0 con 0x7fb1480725f0 2024-01-21T22:24:33.345 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.343+0000 7fb13effd640 1 -- 172.21.15.152:0/3805540234 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7fb130039cf0 con 0x7fb1480725f0 2024-01-21T22:24:33.421 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:33.419+0000 7f8d15c41040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-01-21T22:24:33.426 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.425+0000 7fb14cea6640 1 -- 172.21.15.152:0/3805540234 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4bd94373-76ca-47a6-870b-16d96d7c9e81", "id": 2} v 0) v1 -- 0x7fb148066620 con 0x7fb1480725f0 2024-01-21T22:24:33.587 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:33.585+0000 7f8d15c41040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-01-21T22:24:33.600 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.599+0000 7fb13effd640 1 -- 172.21.15.152:0/3805540234 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4bd94373-76ca-47a6-870b-16d96d7c9e81", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb130028020 con 0x7fb1480725f0 2024-01-21T22:24:33.600 INFO:teuthology.orchestra.run.smithi152.stdout:2 2024-01-21T22:24:33.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.600+0000 7fb14cea6640 1 -- 172.21.15.152:0/3805540234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1480725f0 msgr2=0x7fb1480c11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:33.601 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.600+0000 7fb14cea6640 1 --2- 172.21.15.152:0/3805540234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1480725f0 0x7fb1480c11b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb13000b3e0 tx=0x7fb13000b8a0 comp rx=0 tx=0).stop 2024-01-21T22:24:33.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.600+0000 7fb14cea6640 1 -- 172.21.15.152:0/3805540234 shutdown_connections 2024-01-21T22:24:33.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.600+0000 7fb14cea6640 1 --2- 172.21.15.152:0/3805540234 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb148151e70 0x7fb1480c16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:33.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.600+0000 7fb14cea6640 1 --2- 172.21.15.152:0/3805540234 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb1480725f0 0x7fb1480c11b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:33.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.600+0000 7fb14cea6640 1 --2- 172.21.15.152:0/3805540234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb14806c070 0x7fb14814ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:33.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.600+0000 7fb14cea6640 1 -- 172.21.15.152:0/3805540234 >> 172.21.15.152:0/3805540234 conn(0x7fb1480cb7d0 msgr2=0x7fb1480cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:33.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.600+0000 7fb14cea6640 1 -- 172.21.15.152:0/3805540234 shutdown_connections 2024-01-21T22:24:33.602 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.600+0000 7fb14cea6640 1 -- 172.21.15.152:0/3805540234 wait complete. 2024-01-21T22:24:33.613 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 07267087-d4d2-4e88-ab9b-bb87f6dd061a 3 2024-01-21T22:24:33.639 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:33.637+0000 7f8d15c41040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-01-21T22:24:33.754 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.753+0000 7f1ccdddc640 1 Processor -- start 2024-01-21T22:24:33.755 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.753+0000 7f1ccdddc640 1 -- start start 2024-01-21T22:24:33.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.754+0000 7f1ccdddc640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1cc8153e40 0x7f1cc806a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:33.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.754+0000 7f1ccdddc640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1cc80c6160 0x7f1cc80c6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:33.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.754+0000 7f1ccdddc640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc80711c0 con 0x7f1cc8071ef0 2024-01-21T22:24:33.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.754+0000 7f1ccdddc640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc80723a0 con 0x7f1cc80c6160 2024-01-21T22:24:33.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.754+0000 7f1ccdddc640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc8154220 con 0x7f1cc8153e40 2024-01-21T22:24:33.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.755+0000 7f1cc77fe640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1cc8153e40 0x7f1cc806a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:33.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.755+0000 7f1cc7fff640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f1cc8071ef0 0x7f1cc80c70a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:60956/0 (socket says 172.21.15.152:60956) 2024-01-21T22:24:33.756 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.755+0000 7f1cc7fff640 1 -- 172.21.15.152:0/1702577356 learned_addr learned my addr 172.21.15.152:0/1702577356 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-01-21T22:24:33.757 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.756+0000 7f1cbeffd640 1 --2- 172.21.15.152:0/1702577356 >> v2:172.21.15.138:3300/0 conn(0x7f1cc80c6160 0x7f1cc80c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:33.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.757+0000 7f1cc6ffd640 1 -- 172.21.15.152:0/1702577356 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2409328698 0 0) 0x7f1cc80711c0 con 0x7f1cc8071ef0 2024-01-21T22:24:33.758 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.757+0000 7f1cc6ffd640 1 -- 172.21.15.152:0/1702577356 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cb4003520 con 0x7f1cc8071ef0 2024-01-21T22:24:33.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.757+0000 7f1cc77fe640 1 -- 172.21.15.152:0/1702577356 >> v2:172.21.15.138:3300/0 conn(0x7f1cc80c6160 msgr2=0x7f1cc80c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:33.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.757+0000 7f1cc77fe640 1 --2- 172.21.15.152:0/1702577356 >> v2:172.21.15.138:3300/0 conn(0x7f1cc80c6160 0x7f1cc80c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:33.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.757+0000 7f1cc77fe640 1 -- 172.21.15.152:0/1702577356 >> v1:172.21.15.152:6789/0 conn(0x7f1cc8071ef0 legacy=0x7f1cc80c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:33.759 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.757+0000 7f1cc77fe640 1 -- 172.21.15.152:0/1702577356 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc815dba0 con 0x7f1cc8153e40 2024-01-21T22:24:33.760 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.759+0000 7f1cc77fe640 1 --2- 172.21.15.152:0/1702577356 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1cc8153e40 0x7f1cc806a6f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80097a0 tx=0x7f1cb802b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92f76b1417529213 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:33.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.763+0000 7f1cc6ffd640 1 -- 172.21.15.152:0/1702577356 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8030610 con 0x7f1cc8153e40 2024-01-21T22:24:33.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.763+0000 7f1cc6ffd640 1 -- 172.21.15.152:0/1702577356 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb8030770 con 0x7f1cc8153e40 2024-01-21T22:24:33.764 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.763+0000 7f1cc6ffd640 1 -- 172.21.15.152:0/1702577356 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8030b00 con 0x7f1cc8153e40 2024-01-21T22:24:33.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.763+0000 7f1ccdddc640 1 -- 172.21.15.152:0/1702577356 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1cc8153e40 msgr2=0x7f1cc806a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:33.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.763+0000 7f1ccdddc640 1 --2- 172.21.15.152:0/1702577356 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1cc8153e40 0x7f1cc806a6f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80097a0 tx=0x7f1cb802b960 comp rx=0 tx=0).stop 2024-01-21T22:24:33.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.763+0000 7f1ccdddc640 1 -- 172.21.15.152:0/1702577356 shutdown_connections 2024-01-21T22:24:33.765 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.763+0000 7f1ccdddc640 1 --2- 172.21.15.152:0/1702577356 >> v2:172.21.15.138:3300/0 conn(0x7f1cc80c6160 0x7f1cc80c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:33.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.763+0000 7f1ccdddc640 1 --2- 172.21.15.152:0/1702577356 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1cc8153e40 0x7f1cc806a6f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:33.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.763+0000 7f1ccdddc640 1 -- 172.21.15.152:0/1702577356 >> 172.21.15.152:0/1702577356 conn(0x7f1cc80c8260 msgr2=0x7f1cc80c8670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:33.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.763+0000 7f1ccdddc640 1 -- 172.21.15.152:0/1702577356 shutdown_connections 2024-01-21T22:24:33.766 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.764+0000 7f1ccdddc640 1 -- 172.21.15.152:0/1702577356 wait complete. 2024-01-21T22:24:33.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.765+0000 7f1ccdddc640 1 Processor -- start 2024-01-21T22:24:33.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.765+0000 7f1ccdddc640 1 -- start start 2024-01-21T22:24:33.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.766+0000 7f1ccdddc640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1cc8071ef0 0x7f1cc8157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:33.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.766+0000 7f1ccdddc640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1cc80c6160 0x7f1cc806e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:33.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.766+0000 7f1ccdddc640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cc8153e40 0x7f1cc80bdd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:33.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.766+0000 7f1ccdddc640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc8159680 con 0x7f1cc8153e40 2024-01-21T22:24:33.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.766+0000 7f1ccdddc640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc8158b90 con 0x7f1cc8071ef0 2024-01-21T22:24:33.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.766+0000 7f1ccdddc640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc8158de0 con 0x7f1cc80c6160 2024-01-21T22:24:33.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.766+0000 7f1cbeffd640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1cc80c6160 0x7f1cc806e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:33.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.766+0000 7f1cc77fe640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1cc8071ef0 0x7f1cc8157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:33.768 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.767+0000 7f1cbeffd640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1cc80c6160 0x7f1cc806e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.152:36548/0 (socket says 172.21.15.152:36548) 2024-01-21T22:24:33.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.767+0000 7f1cbeffd640 1 -- 172.21.15.152:0/758299105 learned_addr learned my addr 172.21.15.152:0/758299105 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-01-21T22:24:33.769 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.768+0000 7f1cc7fff640 1 --2- 172.21.15.152:0/758299105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cc8153e40 0x7f1cc80bdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:33.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.770+0000 7f1cbeffd640 1 -- 172.21.15.152:0/758299105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1cc8071ef0 msgr2=0x7f1cc8157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:33.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.770+0000 7f1cbeffd640 1 --2- 172.21.15.152:0/758299105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1cc8071ef0 0x7f1cc8157dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:33.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.770+0000 7f1cbeffd640 1 -- 172.21.15.152:0/758299105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cc8153e40 msgr2=0x7f1cc80bdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:33.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.770+0000 7f1cbeffd640 1 --2- 172.21.15.152:0/758299105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cc8153e40 0x7f1cc80bdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:33.771 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.770+0000 7f1cbeffd640 1 -- 172.21.15.152:0/758299105 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb8009400 con 0x7f1cc80c6160 2024-01-21T22:24:33.773 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.772+0000 7f1cbeffd640 1 --2- 172.21.15.152:0/758299105 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1cc80c6160 0x7f1cc806e9a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000d5f0 tx=0x7f1cb000dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:33.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.775+0000 7f1cc4ff9640 1 -- 172.21.15.152:0/758299105 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb00125f0 con 0x7f1cc80c6160 2024-01-21T22:24:33.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.775+0000 7f1cc4ff9640 1 -- 172.21.15.152:0/758299105 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb0012750 con 0x7f1cc80c6160 2024-01-21T22:24:33.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.775+0000 7f1ccdddc640 1 -- 172.21.15.152:0/758299105 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc81590c0 con 0x7f1cc80c6160 2024-01-21T22:24:33.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.775+0000 7f1cc4ff9640 1 -- 172.21.15.152:0/758299105 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0012b00 con 0x7f1cc80c6160 2024-01-21T22:24:33.777 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.775+0000 7f1ccdddc640 1 -- 172.21.15.152:0/758299105 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc81578b0 con 0x7f1cc80c6160 2024-01-21T22:24:33.778 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.776+0000 7f1ccdddc640 1 -- 172.21.15.152:0/758299105 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc8157a70 con 0x7f1cc80c6160 2024-01-21T22:24:33.797 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:33.794+0000 7f8d15c41040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-01-21T22:24:33.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.880+0000 7f1cc4ff9640 1 -- 172.21.15.152:0/758299105 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7f1cb001e460 con 0x7f1cc80c6160 2024-01-21T22:24:33.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.880+0000 7f1cc4ff9640 1 -- 172.21.15.152:0/758299105 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f1cb0028cd0 con 0x7f1cc80c6160 2024-01-21T22:24:33.881 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.880+0000 7f1cc4ff9640 1 -- 172.21.15.152:0/758299105 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7f1cb0028840 con 0x7f1cc80c6160 2024-01-21T22:24:33.992 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:33.991+0000 7f1ccdddc640 1 -- 172.21.15.152:0/758299105 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd new", "uuid": "07267087-d4d2-4e88-ab9b-bb87f6dd061a", "id": 3} v 0) v1 -- 0x7f1cc8157710 con 0x7f1cc80c6160 2024-01-21T22:24:34.113 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.112+0000 7f1cc4ff9640 1 -- 172.21.15.152:0/758299105 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "07267087-d4d2-4e88-ab9b-bb87f6dd061a", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1cc8157710 con 0x7f1cc80c6160 2024-01-21T22:24:34.114 INFO:teuthology.orchestra.run.smithi152.stdout:3 2024-01-21T22:24:34.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.113+0000 7f1ccdddc640 1 -- 172.21.15.152:0/758299105 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1cc80c6160 msgr2=0x7f1cc806e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:34.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.113+0000 7f1ccdddc640 1 --2- 172.21.15.152:0/758299105 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1cc80c6160 0x7f1cc806e9a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000d5f0 tx=0x7f1cb000dab0 comp rx=0 tx=0).stop 2024-01-21T22:24:34.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.113+0000 7f1ccdddc640 1 -- 172.21.15.152:0/758299105 shutdown_connections 2024-01-21T22:24:34.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.113+0000 7f1ccdddc640 1 --2- 172.21.15.152:0/758299105 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cc8153e40 0x7f1cc80bdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:34.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.113+0000 7f1ccdddc640 1 --2- 172.21.15.152:0/758299105 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1cc80c6160 0x7f1cc806e9a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:34.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.113+0000 7f1ccdddc640 1 --2- 172.21.15.152:0/758299105 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1cc8071ef0 0x7f1cc8157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:34.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.113+0000 7f1ccdddc640 1 -- 172.21.15.152:0/758299105 >> 172.21.15.152:0/758299105 conn(0x7f1cc80c8260 msgr2=0x7f1cc8152730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:34.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.113+0000 7f1ccdddc640 1 -- 172.21.15.152:0/758299105 shutdown_connections 2024-01-21T22:24:34.115 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.113+0000 7f1ccdddc640 1 -- 172.21.15.152:0/758299105 wait complete. 2024-01-21T22:24:34.126 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new b47ef758-d783-4168-a9a5-11db3de4ca12 4 2024-01-21T22:24:34.127 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:34.125+0000 7fc104e38040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-21T22:24:34.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.227+0000 7f8fb3589640 1 Processor -- start 2024-01-21T22:24:34.229 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.228+0000 7f8fb3589640 1 -- start start 2024-01-21T22:24:34.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.229+0000 7f8fb3589640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8fac071d00 0x7f8fac0720e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:34.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.229+0000 7f8fb3589640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8fac0726e0 0x7f8fac0cc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:34.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.229+0000 7f8fb3589640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8fac0712c0 0x7f8fac0cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:34.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.229+0000 7f8fb3589640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fac157530 con 0x7f8fac071d00 2024-01-21T22:24:34.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.229+0000 7f8fb3589640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fac072bd0 con 0x7f8fac0712c0 2024-01-21T22:24:34.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.229+0000 7f8fb3589640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac156c00 con 0x7f8fac0726e0 2024-01-21T22:24:34.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.230+0000 7f8fb0b24640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8fac0726e0 0x7f8fac0cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:34.231 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.230+0000 7f8fb0b24640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8fac0726e0 0x7f8fac0cc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.152:36556/0 (socket says 172.21.15.152:36556) 2024-01-21T22:24:34.232 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.230+0000 7f8fb0b24640 1 -- 172.21.15.152:0/1822354772 learned_addr learned my addr 172.21.15.152:0/1822354772 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-01-21T22:24:34.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.232+0000 7f8fb0b24640 1 -- 172.21.15.152:0/1822354772 >> v2:172.21.15.152:3300/0 conn(0x7f8fac0712c0 msgr2=0x7f8fac0cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:34.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.232+0000 7f8fb0b24640 1 --2- 172.21.15.152:0/1822354772 >> v2:172.21.15.152:3300/0 conn(0x7f8fac0712c0 0x7f8fac0cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:34.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.232+0000 7f8fb0b24640 1 -- 172.21.15.152:0/1822354772 >> v2:172.21.15.138:3300/0 conn(0x7f8fac071d00 msgr2=0x7f8fac0720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:34.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.232+0000 7f8fb0b24640 1 --2- 172.21.15.152:0/1822354772 >> v2:172.21.15.138:3300/0 conn(0x7f8fac071d00 0x7f8fac0720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:34.234 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.232+0000 7f8fb0b24640 1 -- 172.21.15.152:0/1822354772 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fac156ee0 con 0x7f8fac0726e0 2024-01-21T22:24:34.242 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.241+0000 7f8fb0b24640 1 --2- 172.21.15.152:0/1822354772 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8fac0726e0 0x7f8fac0cc320 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8fac06c9e0 tx=0x7f8f9c02e0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=177c2702d1da299c server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:34.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.269+0000 7f8fa3fff640 1 -- 172.21.15.152:0/1822354772 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c02ee90 con 0x7f8fac0726e0 2024-01-21T22:24:34.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.269+0000 7f8fa3fff640 1 -- 172.21.15.152:0/1822354772 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9c036440 con 0x7f8fac0726e0 2024-01-21T22:24:34.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.269+0000 7f8fa3fff640 1 -- 172.21.15.152:0/1822354772 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c0367d0 con 0x7f8fac0726e0 2024-01-21T22:24:34.272 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.270+0000 7f8fb3589640 1 -- 172.21.15.152:0/1822354772 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8fac0726e0 msgr2=0x7f8fac0cc320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:34.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.270+0000 7f8fb3589640 1 --2- 172.21.15.152:0/1822354772 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8fac0726e0 0x7f8fac0cc320 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8fac06c9e0 tx=0x7f8f9c02e0e0 comp rx=0 tx=0).stop 2024-01-21T22:24:34.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.270+0000 7f8fb3589640 1 -- 172.21.15.152:0/1822354772 shutdown_connections 2024-01-21T22:24:34.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.270+0000 7f8fb3589640 1 --2- 172.21.15.152:0/1822354772 >> v2:172.21.15.152:3300/0 conn(0x7f8fac0712c0 0x7f8fac0cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:34.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.270+0000 7f8fb3589640 1 --2- 172.21.15.152:0/1822354772 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8fac0726e0 0x7f8fac0cc320 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:34.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.270+0000 7f8fb3589640 1 --2- 172.21.15.152:0/1822354772 >> v2:172.21.15.138:3300/0 conn(0x7f8fac071d00 0x7f8fac0720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:34.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.270+0000 7f8fb3589640 1 -- 172.21.15.152:0/1822354772 >> 172.21.15.152:0/1822354772 conn(0x7f8fac066400 msgr2=0x7f8fac0667e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:34.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.270+0000 7f8fb3589640 1 -- 172.21.15.152:0/1822354772 shutdown_connections 2024-01-21T22:24:34.273 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.270+0000 7f8fb3589640 1 -- 172.21.15.152:0/1822354772 wait complete. 2024-01-21T22:24:34.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.271+0000 7f8fb3589640 1 Processor -- start 2024-01-21T22:24:34.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.271+0000 7f8fb3589640 1 -- start start 2024-01-21T22:24:34.274 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.272+0000 7f8fb3589640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8fac0712c0 0x7f8fac153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:34.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.272+0000 7f8fb3589640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fac071d00 0x7f8fac151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:34.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.272+0000 7f8fb3589640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fac0726e0 0x7f8fac14d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:34.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.272+0000 7f8fb3589640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac151f40 con 0x7f8fac0726e0 2024-01-21T22:24:34.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.272+0000 7f8fb3589640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac1f2e00 con 0x7f8fac071d00 2024-01-21T22:24:34.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.272+0000 7f8fb3589640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fac151760 con 0x7f8fac0712c0 2024-01-21T22:24:34.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.272+0000 7f8fb1325640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8fac0712c0 0x7f8fac153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:34.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.272+0000 7f8fb0b24640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fac071d00 0x7f8fac151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:34.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.273+0000 7f8fb1325640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8fac0712c0 0x7f8fac153e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.152:36558/0 (socket says 172.21.15.152:36558) 2024-01-21T22:24:34.275 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.273+0000 7f8fb1325640 1 -- 172.21.15.152:0/155138102 learned_addr learned my addr 172.21.15.152:0/155138102 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-01-21T22:24:34.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.277+0000 7f8fb0b24640 1 -- 172.21.15.152:0/155138102 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8fac0712c0 msgr2=0x7f8fac153e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:34.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.277+0000 7f8fb0b24640 1 --2- 172.21.15.152:0/155138102 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8fac0712c0 0x7f8fac153e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:34.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.277+0000 7f8fb0b24640 1 -- 172.21.15.152:0/155138102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fac0726e0 msgr2=0x7f8fac14d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:34.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.277+0000 7f8fb0b24640 1 --2- 172.21.15.152:0/155138102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fac0726e0 0x7f8fac14d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:34.279 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.278+0000 7f8fb0b24640 1 -- 172.21.15.152:0/155138102 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9c00b770 con 0x7f8fac071d00 2024-01-21T22:24:34.281 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.280+0000 7f8fb0b24640 1 --2- 172.21.15.152:0/155138102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fac071d00 0x7f8fac151220 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c035000 tx=0x7f8f9c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:34.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.284+0000 7f8fa1ffb640 1 -- 172.21.15.152:0/155138102 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c034950 con 0x7f8fac071d00 2024-01-21T22:24:34.285 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.284+0000 7f8fa1ffb640 1 -- 172.21.15.152:0/155138102 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9c034ad0 con 0x7f8fac071d00 2024-01-21T22:24:34.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.284+0000 7f8fa1ffb640 1 -- 172.21.15.152:0/155138102 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c034e80 con 0x7f8fac071d00 2024-01-21T22:24:34.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.284+0000 7f8fb3589640 1 -- 172.21.15.152:0/155138102 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fac1f3660 con 0x7f8fac071d00 2024-01-21T22:24:34.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.284+0000 7f8fb3589640 1 -- 172.21.15.152:0/155138102 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fac151980 con 0x7f8fac071d00 2024-01-21T22:24:34.286 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.285+0000 7f8fb3589640 1 -- 172.21.15.152:0/155138102 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fac1f37f0 con 0x7f8fac071d00 2024-01-21T22:24:34.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.288+0000 7f8fa1ffb640 1 -- 172.21.15.152:0/155138102 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7f8f9c041800 con 0x7f8fac071d00 2024-01-21T22:24:34.289 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.288+0000 7f8fa1ffb640 1 -- 172.21.15.152:0/155138102 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f8f9c04bbc0 con 0x7f8fac071d00 2024-01-21T22:24:34.391 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.390+0000 7f8fa1ffb640 1 -- 172.21.15.152:0/155138102 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7f8f9c03b070 con 0x7f8fac071d00 2024-01-21T22:24:34.398 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:24:34.407 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:24:34.423 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:34.422+0000 7fc104e38040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-21T22:24:34.467 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.465+0000 7f8fb3589640 1 -- 172.21.15.152:0/155138102 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b47ef758-d783-4168-a9a5-11db3de4ca12", "id": 4} v 0) v1 -- 0x7f8fac066e50 con 0x7f8fac071d00 2024-01-21T22:24:34.478 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:34.477+0000 7fc104e38040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-21T22:24:34.537 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.536+0000 7f8fa1ffb640 1 -- 172.21.15.152:0/155138102 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b47ef758-d783-4168-a9a5-11db3de4ca12", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8f9c0590b0 con 0x7f8fac071d00 2024-01-21T22:24:34.538 INFO:teuthology.orchestra.run.smithi152.stdout:4 2024-01-21T22:24:34.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.537+0000 7f8fb3589640 1 -- 172.21.15.152:0/155138102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fac071d00 msgr2=0x7f8fac151220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:34.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.537+0000 7f8fb3589640 1 --2- 172.21.15.152:0/155138102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fac071d00 0x7f8fac151220 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c035000 tx=0x7f8f9c004730 comp rx=0 tx=0).stop 2024-01-21T22:24:34.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.537+0000 7f8fb3589640 1 -- 172.21.15.152:0/155138102 shutdown_connections 2024-01-21T22:24:34.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.537+0000 7f8fb3589640 1 --2- 172.21.15.152:0/155138102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fac0726e0 0x7f8fac14d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:34.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.537+0000 7f8fb3589640 1 --2- 172.21.15.152:0/155138102 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8fac071d00 0x7f8fac151220 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:34.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.537+0000 7f8fb3589640 1 --2- 172.21.15.152:0/155138102 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8fac0712c0 0x7f8fac153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:34.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.537+0000 7f8fb3589640 1 -- 172.21.15.152:0/155138102 >> 172.21.15.152:0/155138102 conn(0x7f8fac066400 msgr2=0x7f8fac0cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:34.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.537+0000 7f8fb3589640 1 -- 172.21.15.152:0/155138102 shutdown_connections 2024-01-21T22:24:34.539 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.537+0000 7f8fb3589640 1 -- 172.21.15.152:0/155138102 wait complete. 2024-01-21T22:24:34.549 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 0ea5d43e-61d8-4823-815f-9e42c62fae97 5 2024-01-21T22:24:34.588 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:34.586+0000 7fc104e38040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-21T22:24:34.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.685+0000 7fc5db3a9640 1 Processor -- start 2024-01-21T22:24:34.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.686+0000 7fc5db3a9640 1 -- start start 2024-01-21T22:24:34.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.686+0000 7fc5db3a9640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc5d415a900 0x7fc5d40cd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:34.687 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.686+0000 7fc5db3a9640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d40c72d0 con 0x7fc5d406eca0 2024-01-21T22:24:34.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.686+0000 7fc5db3a9640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d406b8a0 con 0x7fc5d40c8b40 2024-01-21T22:24:34.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.686+0000 7fc5db3a9640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d40c6fa0 con 0x7fc5d415a900 2024-01-21T22:24:34.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.687+0000 7fc5d9145640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc5d415a900 0x7fc5d40cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:34.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.687+0000 7fc5d9145640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc5d415a900 0x7fc5d40cd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50852/0 (socket says 172.21.15.152:50852) 2024-01-21T22:24:34.688 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.687+0000 7fc5d9145640 1 -- 172.21.15.152:0/2506719025 learned_addr learned my addr 172.21.15.152:0/2506719025 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-01-21T22:24:34.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.689+0000 7fc5d9145640 1 -- 172.21.15.152:0/2506719025 >> v1:172.21.15.152:6789/0 conn(0x7fc5d40c8b40 legacy=0x7fc5d4164580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:34.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.689+0000 7fc5d9145640 1 -- 172.21.15.152:0/2506719025 >> v1:172.21.15.138:6789/0 conn(0x7fc5d406eca0 legacy=0x7fc5d40c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:34.691 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.690+0000 7fc5d9145640 1 -- 172.21.15.152:0/2506719025 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d406bb40 con 0x7fc5d415a900 2024-01-21T22:24:34.692 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.691+0000 7fc5d9145640 1 --2- 172.21.15.152:0/2506719025 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5d415a900 0x7fc5d40cd100 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009870 tx=0x7fc5c402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0736956904678b0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:34.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.695+0000 7fc5cbfff640 1 -- 172.21.15.152:0/2506719025 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4031870 con 0x7fc5d415a900 2024-01-21T22:24:34.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.696+0000 7fc5cbfff640 1 -- 172.21.15.152:0/2506719025 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c40319d0 con 0x7fc5d415a900 2024-01-21T22:24:34.697 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.696+0000 7fc5cbfff640 1 -- 172.21.15.152:0/2506719025 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4031d60 con 0x7fc5d415a900 2024-01-21T22:24:34.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.696+0000 7fc5db3a9640 1 -- 172.21.15.152:0/2506719025 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5d415a900 msgr2=0x7fc5d40cd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:34.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.696+0000 7fc5db3a9640 1 --2- 172.21.15.152:0/2506719025 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5d415a900 0x7fc5d40cd100 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009870 tx=0x7fc5c402b910 comp rx=0 tx=0).stop 2024-01-21T22:24:34.698 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.696+0000 7fc5db3a9640 1 -- 172.21.15.152:0/2506719025 shutdown_connections 2024-01-21T22:24:34.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.696+0000 7fc5db3a9640 1 --2- 172.21.15.152:0/2506719025 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5d415a900 0x7fc5d40cd100 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:34.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.696+0000 7fc5db3a9640 1 -- 172.21.15.152:0/2506719025 >> 172.21.15.152:0/2506719025 conn(0x7fc5d406a8d0 msgr2=0x7fc5d4066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:34.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.696+0000 7fc5db3a9640 1 -- 172.21.15.152:0/2506719025 shutdown_connections 2024-01-21T22:24:34.699 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.696+0000 7fc5db3a9640 1 -- 172.21.15.152:0/2506719025 wait complete. 2024-01-21T22:24:34.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.698+0000 7fc5db3a9640 1 Processor -- start 2024-01-21T22:24:34.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.698+0000 7fc5db3a9640 1 -- start start 2024-01-21T22:24:34.700 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.698+0000 7fc5db3a9640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5d406eca0 0x7fc5d40cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:34.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.698+0000 7fc5db3a9640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5d40c8b40 0x7fc5d41efc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:34.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.698+0000 7fc5db3a9640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5d415a900 0x7fc5d41f01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:34.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.698+0000 7fc5db3a9640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d41559b0 con 0x7fc5d406eca0 2024-01-21T22:24:34.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.698+0000 7fc5db3a9640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d41ef840 con 0x7fc5d40c8b40 2024-01-21T22:24:34.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.698+0000 7fc5db3a9640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d4155410 con 0x7fc5d415a900 2024-01-21T22:24:34.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.699+0000 7fc5d8944640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5d40c8b40 0x7fc5d41efc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:34.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.699+0000 7fc5d9946640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5d415a900 0x7fc5d41f01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:34.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.699+0000 7fc5d8944640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5d40c8b40 0x7fc5d41efc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50858/0 (socket says 172.21.15.152:50858) 2024-01-21T22:24:34.701 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.699+0000 7fc5d8944640 1 -- 172.21.15.152:0/1989144944 learned_addr learned my addr 172.21.15.152:0/1989144944 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-01-21T22:24:34.703 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.702+0000 7fc5d8944640 1 -- 172.21.15.152:0/1989144944 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5d415a900 msgr2=0x7fc5d41f01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:34.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.702+0000 7fc5d8944640 1 --2- 172.21.15.152:0/1989144944 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5d415a900 0x7fc5d41f01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:34.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.702+0000 7fc5d8944640 1 -- 172.21.15.152:0/1989144944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5d406eca0 msgr2=0x7fc5d40cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:34.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.702+0000 7fc5d8944640 1 --2- 172.21.15.152:0/1989144944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5d406eca0 0x7fc5d40cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:34.704 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.702+0000 7fc5d8944640 1 -- 172.21.15.152:0/1989144944 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c40094d0 con 0x7fc5d40c8b40 2024-01-21T22:24:34.706 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.705+0000 7fc5d8944640 1 --2- 172.21.15.152:0/1989144944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5d40c8b40 0x7fc5d41efc90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc5d000c420 tx=0x7fc5d000c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:34.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.708+0000 7fc5c9ffb640 1 -- 172.21.15.152:0/1989144944 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d00035f0 con 0x7fc5d40c8b40 2024-01-21T22:24:34.709 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.708+0000 7fc5c9ffb640 1 -- 172.21.15.152:0/1989144944 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d0003750 con 0x7fc5d40c8b40 2024-01-21T22:24:34.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.708+0000 7fc5c9ffb640 1 -- 172.21.15.152:0/1989144944 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d0003b00 con 0x7fc5d40c8b40 2024-01-21T22:24:34.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.708+0000 7fc5db3a9640 1 -- 172.21.15.152:0/1989144944 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d40c6a60 con 0x7fc5d40c8b40 2024-01-21T22:24:34.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.708+0000 7fc5db3a9640 1 -- 172.21.15.152:0/1989144944 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d4151e40 con 0x7fc5d40c8b40 2024-01-21T22:24:34.710 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.709+0000 7fc5db3a9640 1 -- 172.21.15.152:0/1989144944 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d4151c80 con 0x7fc5d40c8b40 2024-01-21T22:24:34.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.808+0000 7fc5c9ffb640 1 -- 172.21.15.152:0/1989144944 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 1) v1 ==== 799+0+0 (secure 0 0 0) 0x7fc5d0007030 con 0x7fc5d40c8b40 2024-01-21T22:24:34.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.808+0000 7fc5c9ffb640 1 -- 172.21.15.152:0/1989144944 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fc5d0012980 con 0x7fc5d40c8b40 2024-01-21T22:24:34.810 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.808+0000 7fc5c9ffb640 1 -- 172.21.15.152:0/1989144944 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7fc5d0017070 con 0x7fc5d40c8b40 2024-01-21T22:24:34.813 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:34.811+0000 7f8d15c41040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-01-21T22:24:34.880 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:34.879+0000 7fc5db3a9640 1 -- 172.21.15.152:0/1989144944 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0ea5d43e-61d8-4823-815f-9e42c62fae97", "id": 5} v 0) v1 -- 0x7fc5d406cca0 con 0x7fc5d40c8b40 2024-01-21T22:24:35.150 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:35.148+0000 7f8d15c41040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-01-21T22:24:35.210 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:35.208+0000 7f8d15c41040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-01-21T22:24:35.271 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.270+0000 7fc5c9ffb640 1 -- 172.21.15.152:0/1989144944 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mgrmap(e 2) v1 ==== 44779+0+0 (secure 0 0 0) 0x7fc5d00074c0 con 0x7fc5d40c8b40 2024-01-21T22:24:35.333 INFO:tasks.ceph.mgr.y.smithi138.stderr:2024-01-21T22:24:35.330+0000 7f8d15c41040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-01-21T22:24:35.339 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.338+0000 7fc5c9ffb640 1 -- 172.21.15.152:0/1989144944 <== mon.1 v2:172.21.15.152:3300/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0ea5d43e-61d8-4823-815f-9e42c62fae97", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc5d0010cd0 con 0x7fc5d40c8b40 2024-01-21T22:24:35.339 INFO:teuthology.orchestra.run.smithi152.stdout:5 2024-01-21T22:24:35.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.339+0000 7fc5db3a9640 1 -- 172.21.15.152:0/1989144944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5d40c8b40 msgr2=0x7fc5d41efc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:35.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.339+0000 7fc5db3a9640 1 --2- 172.21.15.152:0/1989144944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5d40c8b40 0x7fc5d41efc90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc5d000c420 tx=0x7fc5d000c8e0 comp rx=0 tx=0).stop 2024-01-21T22:24:35.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.339+0000 7fc5db3a9640 1 -- 172.21.15.152:0/1989144944 shutdown_connections 2024-01-21T22:24:35.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.339+0000 7fc5db3a9640 1 --2- 172.21.15.152:0/1989144944 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc5d415a900 0x7fc5d41f01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:35.340 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.339+0000 7fc5db3a9640 1 --2- 172.21.15.152:0/1989144944 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc5d40c8b40 0x7fc5d41efc90 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:35.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.339+0000 7fc5db3a9640 1 --2- 172.21.15.152:0/1989144944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5d406eca0 0x7fc5d40cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:35.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.339+0000 7fc5db3a9640 1 -- 172.21.15.152:0/1989144944 >> 172.21.15.152:0/1989144944 conn(0x7fc5d406a8d0 msgr2=0x7fc5d406e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:35.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.339+0000 7fc5db3a9640 1 -- 172.21.15.152:0/1989144944 shutdown_connections 2024-01-21T22:24:35.341 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.339+0000 7fc5db3a9640 1 -- 172.21.15.152:0/1989144944 wait complete. 2024-01-21T22:24:35.350 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 59ffac36-e2e0-4693-ac82-ad5eccf6224d 6 2024-01-21T22:24:35.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.451+0000 7fc3f0cad640 1 Processor -- start 2024-01-21T22:24:35.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.451+0000 7fc3f0cad640 1 -- start start 2024-01-21T22:24:35.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.451+0000 7fc3f0cad640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fc3ec153c60 0x7fc3ec0c7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:35.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.451+0000 7fc3f0cad640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3ec0c7290 con 0x7fc3ec0c9bd0 2024-01-21T22:24:35.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.451+0000 7fc3f0cad640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3ec15a570 con 0x7fc3ec0c6900 2024-01-21T22:24:35.453 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.451+0000 7fc3f0cad640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3ec0cb640 con 0x7fc3ec153c60 2024-01-21T22:24:35.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.452+0000 7fc3ead9d640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fc3ec0c6900 0x7fc3ec1642f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54080/0 (socket says 172.21.15.152:54080) 2024-01-21T22:24:35.454 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.452+0000 7fc3ead9d640 1 -- 172.21.15.152:0/2829742538 learned_addr learned my addr 172.21.15.152:0/2829742538 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-01-21T22:24:35.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.455+0000 7fc3e9d9b640 1 -- 172.21.15.152:0/2829742538 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 464014099 0 0) 0x7fc3ec15a570 con 0x7fc3ec0c6900 2024-01-21T22:24:35.456 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.455+0000 7fc3e9d9b640 1 -- 172.21.15.152:0/2829742538 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3d8003520 con 0x7fc3ec0c6900 2024-01-21T22:24:35.499 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.498+0000 7fc3ea59c640 1 --2- 172.21.15.152:0/2829742538 >> v2:172.21.15.138:3300/0 conn(0x7fc3ec153c60 0x7fc3ec0c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:35.556 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.555+0000 7fc3ea59c640 1 -- 172.21.15.152:0/2829742538 >> v1:172.21.15.152:6789/0 conn(0x7fc3ec0c6900 legacy=0x7fc3ec1642f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:35.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.555+0000 7fc3ea59c640 1 -- 172.21.15.152:0/2829742538 >> v1:172.21.15.138:6789/0 conn(0x7fc3ec0c9bd0 legacy=0x7fc3ec0ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:35.557 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.555+0000 7fc3ea59c640 1 -- 172.21.15.152:0/2829742538 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ec0ca110 con 0x7fc3ec153c60 2024-01-21T22:24:35.648 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.647+0000 7fc3ea59c640 1 --2- 172.21.15.152:0/2829742538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3ec153c60 0x7fc3ec0c7720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0097a0 tx=0x7fc3dc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0bddd003f1442c server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:35.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.816+0000 7fc3e9d9b640 1 -- 172.21.15.152:0/2829742538 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc031830 con 0x7fc3ec153c60 2024-01-21T22:24:35.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.816+0000 7fc3e9d9b640 1 -- 172.21.15.152:0/2829742538 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc031990 con 0x7fc3ec153c60 2024-01-21T22:24:35.818 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.816+0000 7fc3e9d9b640 1 -- 172.21.15.152:0/2829742538 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc031d20 con 0x7fc3ec153c60 2024-01-21T22:24:35.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.817+0000 7fc3f0cad640 1 -- 172.21.15.152:0/2829742538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3ec153c60 msgr2=0x7fc3ec0c7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:35.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.817+0000 7fc3f0cad640 1 --2- 172.21.15.152:0/2829742538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3ec153c60 0x7fc3ec0c7720 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0097a0 tx=0x7fc3dc02b8d0 comp rx=0 tx=0).stop 2024-01-21T22:24:35.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.817+0000 7fc3f0cad640 1 -- 172.21.15.152:0/2829742538 shutdown_connections 2024-01-21T22:24:35.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.817+0000 7fc3f0cad640 1 --2- 172.21.15.152:0/2829742538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3ec153c60 0x7fc3ec0c7720 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:35.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.817+0000 7fc3f0cad640 1 -- 172.21.15.152:0/2829742538 >> 172.21.15.152:0/2829742538 conn(0x7fc3ec0cf430 msgr2=0x7fc3ec0cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:35.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.817+0000 7fc3f0cad640 1 -- 172.21.15.152:0/2829742538 shutdown_connections 2024-01-21T22:24:35.819 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.817+0000 7fc3f0cad640 1 -- 172.21.15.152:0/2829742538 wait complete. 2024-01-21T22:24:35.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.819+0000 7fc3f0cad640 1 Processor -- start 2024-01-21T22:24:35.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.819+0000 7fc3f0cad640 1 -- start start 2024-01-21T22:24:35.820 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.819+0000 7fc3f0cad640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3ec0c6900 0x7fc3ec1f0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:35.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.819+0000 7fc3f0cad640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc3ec0c9bd0 0x7fc3ec14f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:35.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.819+0000 7fc3f0cad640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3ec153c60 0x7fc3ec155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:35.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.819+0000 7fc3f0cad640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec151ff0 con 0x7fc3ec153c60 2024-01-21T22:24:35.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.819+0000 7fc3f0cad640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec156e30 con 0x7fc3ec0c6900 2024-01-21T22:24:35.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.819+0000 7fc3f0cad640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec1f0590 con 0x7fc3ec0c9bd0 2024-01-21T22:24:35.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.820+0000 7fc3ea59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3ec0c6900 0x7fc3ec1f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:35.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.820+0000 7fc3ea59c640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3ec0c6900 0x7fc3ec1f0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50946/0 (socket says 172.21.15.152:50946) 2024-01-21T22:24:35.821 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.820+0000 7fc3ea59c640 1 -- 172.21.15.152:0/3957431219 learned_addr learned my addr 172.21.15.152:0/3957431219 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-01-21T22:24:35.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.825+0000 7fc3ea59c640 1 -- 172.21.15.152:0/3957431219 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc3ec0c9bd0 msgr2=0x7fc3ec14f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:35.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.825+0000 7fc3ea59c640 1 --2- 172.21.15.152:0/3957431219 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc3ec0c9bd0 0x7fc3ec14f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:35.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.825+0000 7fc3ea59c640 1 -- 172.21.15.152:0/3957431219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3ec153c60 msgr2=0x7fc3ec155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:35.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.825+0000 7fc3ea59c640 1 --2- 172.21.15.152:0/3957431219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3ec153c60 0x7fc3ec155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:35.826 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.825+0000 7fc3ea59c640 1 -- 172.21.15.152:0/3957431219 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3dc009400 con 0x7fc3ec0c6900 2024-01-21T22:24:35.829 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.829+0000 7fc3ea59c640 1 --2- 172.21.15.152:0/3957431219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3ec0c6900 0x7fc3ec1f0050 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0098c0 tx=0x7fc3dc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:35.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.833+0000 7fc3e37fe640 1 -- 172.21.15.152:0/3957431219 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc03a8f0 con 0x7fc3ec0c6900 2024-01-21T22:24:35.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.833+0000 7fc3e37fe640 1 -- 172.21.15.152:0/3957431219 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc03aa50 con 0x7fc3ec0c6900 2024-01-21T22:24:35.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.833+0000 7fc3f0cad640 1 -- 172.21.15.152:0/3957431219 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3ec1541c0 con 0x7fc3ec0c6900 2024-01-21T22:24:35.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.833+0000 7fc3e37fe640 1 -- 172.21.15.152:0/3957431219 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc03ae00 con 0x7fc3ec0c6900 2024-01-21T22:24:35.834 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.833+0000 7fc3f0cad640 1 -- 172.21.15.152:0/3957431219 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3ec151d10 con 0x7fc3ec0c6900 2024-01-21T22:24:35.835 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.834+0000 7fc3f0cad640 1 -- 172.21.15.152:0/3957431219 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3ec0c8910 con 0x7fc3ec0c6900 2024-01-21T22:24:35.943 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.942+0000 7fc3e37fe640 1 -- 172.21.15.152:0/3957431219 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 2) v1 ==== 44779+0+0 (secure 0 0 0) 0x7fc3dc04d450 con 0x7fc3ec0c6900 2024-01-21T22:24:35.948 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.947+0000 7fc3e37fe640 1 -- 172.21.15.152:0/3957431219 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fc3dc02f2e0 con 0x7fc3ec0c6900 2024-01-21T22:24:35.949 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:35.947+0000 7fc3e37fe640 1 -- 172.21.15.152:0/3957431219 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7fc3dc0433a0 con 0x7fc3ec0c6900 2024-01-21T22:24:36.018 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.016+0000 7fc3f0cad640 1 -- 172.21.15.152:0/3957431219 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd new", "uuid": "59ffac36-e2e0-4693-ac82-ad5eccf6224d", "id": 6} v 0) v1 -- 0x7fc3ec152130 con 0x7fc3ec0c6900 2024-01-21T22:24:36.518 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.516+0000 7fc3e37fe640 1 -- 172.21.15.152:0/3957431219 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "59ffac36-e2e0-4693-ac82-ad5eccf6224d", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc3dc04de60 con 0x7fc3ec0c6900 2024-01-21T22:24:36.518 INFO:teuthology.orchestra.run.smithi152.stdout:6 2024-01-21T22:24:36.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.517+0000 7fc3f0cad640 1 -- 172.21.15.152:0/3957431219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3ec0c6900 msgr2=0x7fc3ec1f0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:36.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.517+0000 7fc3f0cad640 1 --2- 172.21.15.152:0/3957431219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3ec0c6900 0x7fc3ec1f0050 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0098c0 tx=0x7fc3dc0046c0 comp rx=0 tx=0).stop 2024-01-21T22:24:36.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.517+0000 7fc3f0cad640 1 -- 172.21.15.152:0/3957431219 shutdown_connections 2024-01-21T22:24:36.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.517+0000 7fc3f0cad640 1 --2- 172.21.15.152:0/3957431219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3ec153c60 0x7fc3ec155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:36.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.517+0000 7fc3f0cad640 1 --2- 172.21.15.152:0/3957431219 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc3ec0c9bd0 0x7fc3ec14f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:36.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.517+0000 7fc3f0cad640 1 --2- 172.21.15.152:0/3957431219 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc3ec0c6900 0x7fc3ec1f0050 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:36.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.517+0000 7fc3f0cad640 1 -- 172.21.15.152:0/3957431219 >> 172.21.15.152:0/3957431219 conn(0x7fc3ec0cf430 msgr2=0x7fc3ec152b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:36.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.517+0000 7fc3f0cad640 1 -- 172.21.15.152:0/3957431219 shutdown_connections 2024-01-21T22:24:36.519 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.517+0000 7fc3f0cad640 1 -- 172.21.15.152:0/3957431219 wait complete. 2024-01-21T22:24:36.532 DEBUG:teuthology.orchestra.run.smithi152:> sudo ceph --cluster ceph osd new 4f892c7d-32a5-4960-bf88-4ed7c863cd0a 7 2024-01-21T22:24:36.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.642+0000 7effab989640 1 Processor -- start 2024-01-21T22:24:36.643 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.642+0000 7effab989640 1 -- start start 2024-01-21T22:24:36.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.643+0000 7effab989640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7effa4159ab0 0x7effa41653e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:36.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.643+0000 7effab989640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa415dff0 con 0x7effa40ca590 2024-01-21T22:24:36.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.643+0000 7effab989640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa415d450 con 0x7effa4153df0 2024-01-21T22:24:36.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.643+0000 7effab989640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa4159ee0 con 0x7effa4159ab0 2024-01-21T22:24:36.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.643+0000 7effa9725640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7effa4153df0 0x7effa415d8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.152:54084/0 (socket says 172.21.15.152:54084) 2024-01-21T22:24:36.644 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.643+0000 7effa9725640 1 -- 172.21.15.152:0/955914835 learned_addr learned my addr 172.21.15.152:0/955914835 (peer_addr_for_me v1:172.21.15.152:0/0) 2024-01-21T22:24:36.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.646+0000 7eff9bfff640 1 -- 172.21.15.152:0/955914835 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3106654519 0 0) 0x7effa415d450 con 0x7effa4153df0 2024-01-21T22:24:36.647 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.646+0000 7eff9bfff640 1 -- 172.21.15.152:0/955914835 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff90003520 con 0x7effa4153df0 2024-01-21T22:24:36.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.649+0000 7eff9bfff640 1 -- 172.21.15.152:0/955914835 <== mon.1 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 567951613 0 0) 0x7eff90003520 con 0x7effa4153df0 2024-01-21T22:24:36.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.649+0000 7eff9bfff640 1 -- 172.21.15.152:0/955914835 >> v2:172.21.15.138:3300/0 conn(0x7effa4159ab0 msgr2=0x7effa41653e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:36.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.649+0000 7eff9bfff640 1 --2- 172.21.15.152:0/955914835 >> v2:172.21.15.138:3300/0 conn(0x7effa4159ab0 0x7effa41653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:36.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.649+0000 7eff9bfff640 1 -- 172.21.15.152:0/955914835 >> v1:172.21.15.138:6789/0 conn(0x7effa40ca590 legacy=0x7effa40cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:36.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.649+0000 7eff9bfff640 1 -- 172.21.15.152:0/955914835 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa415a0d0 con 0x7effa4153df0 2024-01-21T22:24:36.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.649+0000 7eff9bfff640 1 -- 172.21.15.152:0/955914835 <== mon.1 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1288616872 0 0) 0x7effa00019f0 con 0x7effa4153df0 2024-01-21T22:24:36.651 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.649+0000 7eff9bfff640 1 -- 172.21.15.152:0/955914835 >> v1:172.21.15.152:6789/0 conn(0x7effa4153df0 legacy=0x7effa415d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.649+0000 7eff9bfff640 1 --2- 172.21.15.152:0/955914835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff900039c0 0x7eff90003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.649+0000 7eff9bfff640 1 --2- 172.21.15.152:0/955914835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff900047c0 0x7eff90004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.649+0000 7eff9bfff640 1 --2- 172.21.15.152:0/955914835 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7eff900053d0 0x7eff9000ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.649+0000 7eff9bfff640 1 -- 172.21.15.152:0/955914835 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9000c2e0 con 0x7eff900039c0 2024-01-21T22:24:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.649+0000 7eff9bfff640 1 -- 172.21.15.152:0/955914835 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9000c450 con 0x7eff900047c0 2024-01-21T22:24:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.649+0000 7eff9bfff640 1 -- 172.21.15.152:0/955914835 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff9000c610 con 0x7eff900053d0 2024-01-21T22:24:36.652 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.651+0000 7effa8f24640 1 --2- 172.21.15.152:0/955914835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff900047c0 0x7eff90004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:36.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.654+0000 7effa8f24640 1 -- 172.21.15.152:0/955914835 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7eff900053d0 msgr2=0x7eff9000ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:36.655 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.654+0000 7effa8f24640 1 --2- 172.21.15.152:0/955914835 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7eff900053d0 0x7eff9000ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:36.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.654+0000 7effa8f24640 1 -- 172.21.15.152:0/955914835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff900039c0 msgr2=0x7eff90003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:36.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.654+0000 7effa8f24640 1 --2- 172.21.15.152:0/955914835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff900039c0 0x7eff90003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:36.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.654+0000 7effa8f24640 1 -- 172.21.15.152:0/955914835 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff90003520 con 0x7eff900047c0 2024-01-21T22:24:36.656 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.654+0000 7effa9f26640 1 -- 172.21.15.152:0/955914835 reap_dead start 2024-01-21T22:24:36.657 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.656+0000 7effa8f24640 1 --2- 172.21.15.152:0/955914835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff900047c0 0x7eff90004c40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7eff94001650 tx=0x7eff900057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3db23e000b6b404 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:36.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.660+0000 7eff9bfff640 1 -- 172.21.15.152:0/955914835 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff94005270 con 0x7eff900047c0 2024-01-21T22:24:36.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.660+0000 7eff9bfff640 1 -- 172.21.15.152:0/955914835 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9402e070 con 0x7eff900047c0 2024-01-21T22:24:36.661 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.660+0000 7eff9bfff640 1 -- 172.21.15.152:0/955914835 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff94003610 con 0x7eff900047c0 2024-01-21T22:24:36.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.660+0000 7effab989640 1 -- 172.21.15.152:0/955914835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff900047c0 msgr2=0x7eff90004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:36.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.660+0000 7effab989640 1 --2- 172.21.15.152:0/955914835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff900047c0 0x7eff90004c40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7eff94001650 tx=0x7eff900057f0 comp rx=0 tx=0).stop 2024-01-21T22:24:36.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.661+0000 7effab989640 1 -- 172.21.15.152:0/955914835 shutdown_connections 2024-01-21T22:24:36.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.661+0000 7effab989640 1 --2- 172.21.15.152:0/955914835 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff900047c0 0x7eff90004c40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:36.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.661+0000 7effab989640 1 -- 172.21.15.152:0/955914835 >> 172.21.15.152:0/955914835 conn(0x7effa406b960 msgr2=0x7effa406cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:36.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.661+0000 7effab989640 1 -- 172.21.15.152:0/955914835 shutdown_connections 2024-01-21T22:24:36.662 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.661+0000 7effab989640 1 -- 172.21.15.152:0/955914835 wait complete. 2024-01-21T22:24:36.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.662+0000 7effab989640 1 Processor -- start 2024-01-21T22:24:36.663 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.662+0000 7effab989640 1 -- start start 2024-01-21T22:24:36.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.662+0000 7effab989640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff900047c0 0x7effa40c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:36.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.663+0000 7effab989640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7effa40c2f40 0x7effa41e4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:36.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.663+0000 7effab989640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effa41f2ca0 0x7effa4158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:36.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.663+0000 7effab989640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa4153000 con 0x7effa41f2ca0 2024-01-21T22:24:36.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.663+0000 7effab989640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa41e5500 con 0x7eff900047c0 2024-01-21T22:24:36.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.663+0000 7effab989640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa4153250 con 0x7effa40c2f40 2024-01-21T22:24:36.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.663+0000 7effa9725640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff900047c0 0x7effa40c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:36.664 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.663+0000 7effa8f24640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7effa40c2f40 0x7effa41e4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:36.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.663+0000 7effa9725640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff900047c0 0x7effa40c37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.152:50964/0 (socket says 172.21.15.152:50964) 2024-01-21T22:24:36.665 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.663+0000 7effa9725640 1 -- 172.21.15.152:0/671545017 learned_addr learned my addr 172.21.15.152:0/671545017 (peer_addr_for_me v2:172.21.15.152:0/0) 2024-01-21T22:24:36.667 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.666+0000 7effa9725640 1 -- 172.21.15.152:0/671545017 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7effa40c2f40 msgr2=0x7effa41e4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:36.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.666+0000 7effa9725640 1 --2- 172.21.15.152:0/671545017 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7effa40c2f40 0x7effa41e4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:36.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.666+0000 7effa9725640 1 -- 172.21.15.152:0/671545017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effa41f2ca0 msgr2=0x7effa4158f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:36.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.666+0000 7effa9725640 1 --2- 172.21.15.152:0/671545017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effa41f2ca0 0x7effa4158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:36.668 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.666+0000 7effa9725640 1 -- 172.21.15.152:0/671545017 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa4153530 con 0x7eff900047c0 2024-01-21T22:24:36.670 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.669+0000 7effa9725640 1 --2- 172.21.15.152:0/671545017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff900047c0 0x7effa40c37c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7effa000caa0 tx=0x7effa000cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:36.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.673+0000 7eff99ffb640 1 -- 172.21.15.152:0/671545017 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa000a790 con 0x7eff900047c0 2024-01-21T22:24:36.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.673+0000 7eff99ffb640 1 -- 172.21.15.152:0/671545017 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa000a8f0 con 0x7eff900047c0 2024-01-21T22:24:36.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.673+0000 7effab989640 1 -- 172.21.15.152:0/671545017 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa41efa60 con 0x7eff900047c0 2024-01-21T22:24:36.674 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.673+0000 7eff99ffb640 1 -- 172.21.15.152:0/671545017 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa000aca0 con 0x7eff900047c0 2024-01-21T22:24:36.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.673+0000 7effab989640 1 -- 172.21.15.152:0/671545017 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa41f0020 con 0x7eff900047c0 2024-01-21T22:24:36.675 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.674+0000 7effab989640 1 -- 172.21.15.152:0/671545017 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa404eb10 con 0x7eff900047c0 2024-01-21T22:24:36.685 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.684+0000 7eff99ffb640 1 -- 172.21.15.152:0/671545017 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 2) v1 ==== 44779+0+0 (secure 0 0 0) 0x7effa0007150 con 0x7eff900047c0 2024-01-21T22:24:36.686 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.685+0000 7eff99ffb640 1 -- 172.21.15.152:0/671545017 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7effa000ae00 con 0x7eff900047c0 2024-01-21T22:24:36.782 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.780+0000 7eff99ffb640 1 -- 172.21.15.152:0/671545017 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72921 (secure 0 0 0) 0x7effa0007400 con 0x7eff900047c0 2024-01-21T22:24:36.862 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:36.861+0000 7effab989640 1 -- 172.21.15.152:0/671545017 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4f892c7d-32a5-4960-bf88-4ed7c863cd0a", "id": 7} v 0) v1 -- 0x7effa40cac50 con 0x7eff900047c0 2024-01-21T22:24:37.357 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:37.356+0000 7fc0a6151640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-01-21T22:24:37.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:37.402+0000 7eff99ffb640 1 -- 172.21.15.152:0/671545017 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mgrmap(e 3) v1 ==== 89354+0+0 (secure 0 0 0) 0x7effa0007780 con 0x7eff900047c0 2024-01-21T22:24:37.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:37.402+0000 7eff99ffb640 1 --2- 172.21.15.152:0/671545017 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7eff8806b640 0x7eff8806da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:37.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:37.403+0000 7effa8f24640 1 --2- 172.21.15.152:0/671545017 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7eff8806b640 0x7eff8806da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:37.404 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:37.403+0000 7effa8f24640 1 --2- 172.21.15.152:0/671545017 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7eff8806b640 0x7eff8806da30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff94033000 tx=0x7eff94009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:37.477 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:24:37.476+0000 7fc0a6151640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-01-21T22:24:37.957 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:37.956+0000 7eff99ffb640 1 -- 172.21.15.152:0/671545017 <== mon.1 v2:172.21.15.152:3300/0 8 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4f892c7d-32a5-4960-bf88-4ed7c863cd0a", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7effa0056ff0 con 0x7eff900047c0 2024-01-21T22:24:38.008 INFO:teuthology.orchestra.run.smithi152.stdout:7 2024-01-21T22:24:38.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:37.957+0000 7effab989640 1 -- 172.21.15.152:0/671545017 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7eff8806b640 msgr2=0x7eff8806da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:38.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:37.957+0000 7effab989640 1 --2- 172.21.15.152:0/671545017 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7eff8806b640 0x7eff8806da30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff94033000 tx=0x7eff94009030 comp rx=0 tx=0).stop 2024-01-21T22:24:38.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:37.957+0000 7effab989640 1 -- 172.21.15.152:0/671545017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff900047c0 msgr2=0x7effa40c37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:38.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:37.957+0000 7effab989640 1 --2- 172.21.15.152:0/671545017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff900047c0 0x7effa40c37c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7effa000caa0 tx=0x7effa000cf60 comp rx=0 tx=0).stop 2024-01-21T22:24:38.009 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:37.957+0000 7effab989640 1 -- 172.21.15.152:0/671545017 shutdown_connections 2024-01-21T22:24:38.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:37.957+0000 7effab989640 1 --2- 172.21.15.152:0/671545017 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7eff8806b640 0x7eff8806da30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:38.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:37.957+0000 7effab989640 1 --2- 172.21.15.152:0/671545017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effa41f2ca0 0x7effa4158f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:38.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:37.957+0000 7effab989640 1 --2- 172.21.15.152:0/671545017 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7effa40c2f40 0x7effa41e4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:38.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:37.957+0000 7effab989640 1 --2- 172.21.15.152:0/671545017 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7eff900047c0 0x7effa40c37c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:38.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:37.957+0000 7effab989640 1 -- 172.21.15.152:0/671545017 >> 172.21.15.152:0/671545017 conn(0x7effa406b960 msgr2=0x7effa414f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:38.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:37.957+0000 7effab989640 1 -- 172.21.15.152:0/671545017 shutdown_connections 2024-01-21T22:24:38.010 INFO:teuthology.orchestra.run.smithi152.stderr:2024-01-21T22:24:37.957+0000 7effab989640 1 -- 172.21.15.152:0/671545017 wait complete. 2024-01-21T22:24:38.010 INFO:tasks.ceph.osd.0:Restarting daemon 2024-01-21T22:24:38.010 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 0 2024-01-21T22:24:38.018 INFO:tasks.ceph.osd.0:Started 2024-01-21T22:24:38.018 INFO:tasks.ceph.osd.1:Restarting daemon 2024-01-21T22:24:38.018 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 1 2024-01-21T22:24:38.026 INFO:tasks.ceph.osd.1:Started 2024-01-21T22:24:38.026 INFO:tasks.ceph.osd.2:Restarting daemon 2024-01-21T22:24:38.026 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 2 2024-01-21T22:24:38.036 INFO:tasks.ceph.osd.2:Started 2024-01-21T22:24:38.036 INFO:tasks.ceph.osd.3:Restarting daemon 2024-01-21T22:24:38.036 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 3 2024-01-21T22:24:38.041 INFO:tasks.ceph.osd.3:Started 2024-01-21T22:24:38.041 INFO:tasks.ceph.osd.4:Restarting daemon 2024-01-21T22:24:38.041 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 4 2024-01-21T22:24:38.043 INFO:tasks.ceph.osd.4:Started 2024-01-21T22:24:38.043 INFO:tasks.ceph.osd.5:Restarting daemon 2024-01-21T22:24:38.044 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 5 2024-01-21T22:24:38.046 INFO:tasks.ceph.osd.5:Started 2024-01-21T22:24:38.046 INFO:tasks.ceph.osd.6:Restarting daemon 2024-01-21T22:24:38.046 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 6 2024-01-21T22:24:38.048 INFO:tasks.ceph.osd.6:Started 2024-01-21T22:24:38.048 INFO:tasks.ceph.osd.7:Restarting daemon 2024-01-21T22:24:38.048 DEBUG:teuthology.orchestra.run.smithi152:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-osd -f --cluster ceph -i 7 2024-01-21T22:24:38.050 INFO:tasks.ceph.osd.7:Started 2024-01-21T22:24:38.050 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-21T22:24:38.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.280+0000 7fdbcd200640 1 Processor -- start 2024-01-21T22:24:38.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.280+0000 7fdbcd200640 1 -- start start 2024-01-21T22:24:38.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.281+0000 7fdbcd200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdbc815a250 0x7fdbc815a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:38.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.281+0000 7fdbcd200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbc80c61c0 con 0x7fdbc8153cc0 2024-01-21T22:24:38.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.282+0000 7fdbcd200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbc8159dc0 con 0x7fdbc806a6f0 2024-01-21T22:24:38.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.282+0000 7fdbcd200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbc8159b20 con 0x7fdbc815a250 2024-01-21T22:24:38.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.282+0000 7fdbbd200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdbc815a250 0x7fdbc815a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:38.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.283+0000 7fdbbd200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdbc815a250 0x7fdbc815a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:58544/0 (socket says 172.21.15.138:58544) 2024-01-21T22:24:38.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.283+0000 7fdbbd200640 1 -- 172.21.15.138:0/1800279850 learned_addr learned my addr 172.21.15.138:0/1800279850 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:24:38.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.285+0000 7fdbbd200640 1 -- 172.21.15.138:0/1800279850 >> v1:172.21.15.152:6789/0 conn(0x7fdbc806a6f0 legacy=0x7fdbc81644a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:38.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.285+0000 7fdbbd200640 1 -- 172.21.15.138:0/1800279850 >> v1:172.21.15.138:6789/0 conn(0x7fdbc8153cc0 legacy=0x7fdbc80c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:38.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.285+0000 7fdbbd200640 1 -- 172.21.15.138:0/1800279850 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc8071270 con 0x7fdbc815a250 2024-01-21T22:24:38.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.287+0000 7fdbbd200640 1 --2- 172.21.15.138:0/1800279850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbc815a250 0x7fdbc815a680 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009870 tx=0x7fdbb002b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=236d5f533565aea1 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:38.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.291+0000 7fdbc5400640 1 -- 172.21.15.138:0/1800279850 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0031870 con 0x7fdbc815a250 2024-01-21T22:24:38.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.291+0000 7fdbc5400640 1 -- 172.21.15.138:0/1800279850 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb00319d0 con 0x7fdbc815a250 2024-01-21T22:24:38.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.291+0000 7fdbc5400640 1 -- 172.21.15.138:0/1800279850 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0031d60 con 0x7fdbc815a250 2024-01-21T22:24:38.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.291+0000 7fdbcd200640 1 -- 172.21.15.138:0/1800279850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbc815a250 msgr2=0x7fdbc815a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:38.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.291+0000 7fdbcd200640 1 --2- 172.21.15.138:0/1800279850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbc815a250 0x7fdbc815a680 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0009870 tx=0x7fdbb002b910 comp rx=0 tx=0).stop 2024-01-21T22:24:38.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.292+0000 7fdbcd200640 1 -- 172.21.15.138:0/1800279850 shutdown_connections 2024-01-21T22:24:38.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.292+0000 7fdbcd200640 1 --2- 172.21.15.138:0/1800279850 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbc815a250 0x7fdbc815a680 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:38.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.292+0000 7fdbcd200640 1 -- 172.21.15.138:0/1800279850 >> 172.21.15.138:0/1800279850 conn(0x7fdbc80cb0b0 msgr2=0x7fdbc80cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:38.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.292+0000 7fdbcd200640 1 -- 172.21.15.138:0/1800279850 shutdown_connections 2024-01-21T22:24:38.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.292+0000 7fdbcd200640 1 -- 172.21.15.138:0/1800279850 wait complete. 2024-01-21T22:24:38.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.293+0000 7fdbcd200640 1 Processor -- start 2024-01-21T22:24:38.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.294+0000 7fdbcd200640 1 -- start start 2024-01-21T22:24:38.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.294+0000 7fdbcd200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbc806a6f0 0x7fdbc8155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:38.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.294+0000 7fdbcd200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbc8153cc0 0x7fdbc81effb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:38.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.294+0000 7fdbcd200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbc815a250 0x7fdbc806c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:38.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.295+0000 7fdbcd200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc81f04f0 con 0x7fdbc806a6f0 2024-01-21T22:24:38.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.295+0000 7fdbcd200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc80c54d0 con 0x7fdbc815a250 2024-01-21T22:24:38.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.295+0000 7fdbcd200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc80c4c50 con 0x7fdbc8153cc0 2024-01-21T22:24:38.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.295+0000 7fdbc6800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbc815a250 0x7fdbc806c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:38.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.295+0000 7fdbbd200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbc8153cc0 0x7fdbc81effb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:38.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.295+0000 7fdbbd200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbc8153cc0 0x7fdbc81effb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:58822/0 (socket says 172.21.15.138:58822) 2024-01-21T22:24:38.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.295+0000 7fdbbd200640 1 -- 172.21.15.138:0/2888087994 learned_addr learned my addr 172.21.15.138:0/2888087994 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:24:38.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.300+0000 7fdbc6800640 1 -- 172.21.15.138:0/2888087994 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbc8153cc0 msgr2=0x7fdbc81effb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:38.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.300+0000 7fdbc6800640 1 --2- 172.21.15.138:0/2888087994 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbc8153cc0 0x7fdbc81effb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:38.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.300+0000 7fdbc6800640 1 -- 172.21.15.138:0/2888087994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbc806a6f0 msgr2=0x7fdbc8155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:38.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.300+0000 7fdbc6800640 1 --2- 172.21.15.138:0/2888087994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbc806a6f0 0x7fdbc8155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:38.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.300+0000 7fdbc6800640 1 -- 172.21.15.138:0/2888087994 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb00094d0 con 0x7fdbc815a250 2024-01-21T22:24:38.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.302+0000 7fdbc6800640 1 --2- 172.21.15.138:0/2888087994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbc815a250 0x7fdbc806c0b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000c410 tx=0x7fdbc000c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:38.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.306+0000 7fdbbea00640 1 -- 172.21.15.138:0/2888087994 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc00035f0 con 0x7fdbc815a250 2024-01-21T22:24:38.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.306+0000 7fdbbea00640 1 -- 172.21.15.138:0/2888087994 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc0003750 con 0x7fdbc815a250 2024-01-21T22:24:38.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.306+0000 7fdbbea00640 1 -- 172.21.15.138:0/2888087994 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0003b00 con 0x7fdbc815a250 2024-01-21T22:24:38.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.306+0000 7fdbcd200640 1 -- 172.21.15.138:0/2888087994 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbc80c4f10 con 0x7fdbc815a250 2024-01-21T22:24:38.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.306+0000 7fdbcd200640 1 -- 172.21.15.138:0/2888087994 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc80c30a0 con 0x7fdbc815a250 2024-01-21T22:24:38.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.307+0000 7fdbcd200640 1 -- 172.21.15.138:0/2888087994 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbc80c9fc0 con 0x7fdbc815a250 2024-01-21T22:24:38.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.326+0000 7fdbbea00640 1 -- 172.21.15.138:0/2888087994 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 3) v1 ==== 89354+0+0 (secure 0 0 0) 0x7fdbc0003c60 con 0x7fdbc815a250 2024-01-21T22:24:38.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.327+0000 7fdbbea00640 1 --2- 172.21.15.138:0/2888087994 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fdba006b1d0 0x7fdba006d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:38.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.327+0000 7fdbc5e00640 1 --2- 172.21.15.138:0/2888087994 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fdba006b1d0 0x7fdba006d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:38.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.327+0000 7fdbbea00640 1 -- 172.21.15.138:0/2888087994 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fdbc0057e80 con 0x7fdbc815a250 2024-01-21T22:24:38.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.328+0000 7fdbc5e00640 1 --2- 172.21.15.138:0/2888087994 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fdba006b1d0 0x7fdba006d690 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8001c60 tx=0x7fdbb8007d10 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:38.653 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-01-21T22:24:38.652+0000 7f078436f640 -1 Falling back to public interface 2024-01-21T22:24:38.658 INFO:tasks.ceph.osd.7.smithi152.stderr:2024-01-21T22:24:38.657+0000 7fea30437640 -1 Falling back to public interface 2024-01-21T22:24:38.709 INFO:tasks.ceph.osd.5.smithi152.stderr:2024-01-21T22:24:38.707+0000 7f4b1dcf9640 -1 Falling back to public interface 2024-01-21T22:24:38.730 INFO:tasks.ceph.osd.4.smithi152.stderr:2024-01-21T22:24:38.729+0000 7f286db38640 -1 Falling back to public interface 2024-01-21T22:24:38.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.748+0000 7fdbbea00640 1 -- 172.21.15.138:0/2888087994 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mgrmap(e 4) v1 ==== 89460+0+0 (secure 0 0 0) 0x7fdbc0054590 con 0x7fdbc815a250 2024-01-21T22:24:38.762 INFO:tasks.ceph.osd.2.smithi138.stderr:2024-01-21T22:24:38.760+0000 7f3a5d39a640 -1 Falling back to public interface 2024-01-21T22:24:38.767 INFO:tasks.ceph.osd.3.smithi138.stderr:2024-01-21T22:24:38.765+0000 7f5d8220c640 -1 Falling back to public interface 2024-01-21T22:24:38.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.773+0000 7fdbbea00640 1 -- 172.21.15.138:0/2888087994 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fdbc007c5a0 con 0x7fdbc815a250 2024-01-21T22:24:38.777 INFO:tasks.ceph.osd.1.smithi138.stderr:2024-01-21T22:24:38.775+0000 7fe1c7e10640 -1 Falling back to public interface 2024-01-21T22:24:38.839 INFO:tasks.ceph.osd.0.smithi138.stderr:2024-01-21T22:24:38.837+0000 7f7a30d93640 -1 Falling back to public interface 2024-01-21T22:24:38.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:38.980+0000 7fdbcd200640 1 -- 172.21.15.138:0/2888087994 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbc804eaa0 con 0x7fdbc815a250 2024-01-21T22:24:39.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:39.030+0000 7fdbbea00640 1 -- 172.21.15.138:0/2888087994 <== mon.1 v2:172.21.15.152:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fdbc00577e0 con 0x7fdbc815a250 2024-01-21T22:24:39.031 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:24:39.032 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":10,"fsid":"d488d446-149f-48d0-89ca-b2830ae72846","created":"2024-01-21T22:24:28.581932+0000","modified":"2024-01-21T22:24:37.672807+0000","last_up_change":"0.000000","last_in_change":"2024-01-21T22:24:37.672807+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d812b2d6-c8a3-4015-bfdb-5766b25eae5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d8d875df-c840-46cc-b6c0-a847f9c93999","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4bd94373-76ca-47a6-870b-16d96d7c9e81","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"07267087-d4d2-4e88-ab9b-bb87f6dd061a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b47ef758-d783-4168-a9a5-11db3de4ca12","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0ea5d43e-61d8-4823-815f-9e42c62fae97","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"59ffac36-e2e0-4693-ac82-ad5eccf6224d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4f892c7d-32a5-4960-bf88-4ed7c863cd0a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-21T22:24:39.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:39.032+0000 7fdbcd200640 1 -- 172.21.15.138:0/2888087994 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fdba006b1d0 msgr2=0x7fdba006d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:39.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:39.032+0000 7fdbcd200640 1 --2- 172.21.15.138:0/2888087994 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fdba006b1d0 0x7fdba006d690 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8001c60 tx=0x7fdbb8007d10 comp rx=0 tx=0).stop 2024-01-21T22:24:39.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:39.032+0000 7fdbcd200640 1 -- 172.21.15.138:0/2888087994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbc815a250 msgr2=0x7fdbc806c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:39.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:39.032+0000 7fdbcd200640 1 --2- 172.21.15.138:0/2888087994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbc815a250 0x7fdbc806c0b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdbc000c410 tx=0x7fdbc000c8d0 comp rx=0 tx=0).stop 2024-01-21T22:24:39.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:39.032+0000 7fdbcd200640 1 -- 172.21.15.138:0/2888087994 shutdown_connections 2024-01-21T22:24:39.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:39.032+0000 7fdbcd200640 1 --2- 172.21.15.138:0/2888087994 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fdba006b1d0 0x7fdba006d690 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:39.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:39.032+0000 7fdbcd200640 1 --2- 172.21.15.138:0/2888087994 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdbc815a250 0x7fdbc806c0b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:39.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:39.032+0000 7fdbcd200640 1 --2- 172.21.15.138:0/2888087994 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdbc8153cc0 0x7fdbc81effb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:39.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:39.032+0000 7fdbcd200640 1 --2- 172.21.15.138:0/2888087994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbc806a6f0 0x7fdbc8155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:39.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:39.032+0000 7fdbcd200640 1 -- 172.21.15.138:0/2888087994 >> 172.21.15.138:0/2888087994 conn(0x7fdbc80cb0b0 msgr2=0x7fdbc80cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:39.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:39.032+0000 7fdbcd200640 1 -- 172.21.15.138:0/2888087994 shutdown_connections 2024-01-21T22:24:39.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:39.032+0000 7fdbcd200640 1 -- 172.21.15.138:0/2888087994 wait complete. 2024-01-21T22:24:39.050 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-01-21T22:24:39.051 INFO:tasks.ceph:Waiting for OSDs to come up 2024-01-21T22:24:39.856 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-21T22:24:40.015 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.014+0000 7fc4a1e00640 1 Processor -- start 2024-01-21T22:24:40.016 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.014+0000 7fc4a1e00640 1 -- start start 2024-01-21T22:24:40.017 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.015+0000 7fc4a1e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc49c153e90 0x7fc49c0cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:40.017 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.016+0000 7fc4a1e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc49c15a950 con 0x7fc49c070ec0 2024-01-21T22:24:40.017 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.016+0000 7fc4a1e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc49c0c9cb0 con 0x7fc49c0712a0 2024-01-21T22:24:40.017 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.016+0000 7fc4a1e00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0cac90 con 0x7fc49c153e90 2024-01-21T22:24:40.017 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.016+0000 7fc4a1400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc49c153e90 0x7fc49c0cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:40.017 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.016+0000 7fc49be00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fc49c0712a0 0x7fc49c15d1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:34866/0 (socket says 172.21.15.138:34866) 2024-01-21T22:24:40.017 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.016+0000 7fc49be00640 1 -- 172.21.15.138:0/2397983227 learned_addr learned my addr 172.21.15.138:0/2397983227 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:24:40.020 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.019+0000 7fc4a1400640 1 -- 172.21.15.138:0/2397983227 >> v1:172.21.15.152:6789/0 conn(0x7fc49c0712a0 legacy=0x7fc49c15d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:40.020 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.019+0000 7fc4a1400640 1 -- 172.21.15.138:0/2397983227 >> v1:172.21.15.138:6789/0 conn(0x7fc49c070ec0 legacy=0x7fc49c153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:40.020 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.019+0000 7fc4a1400640 1 -- 172.21.15.138:0/2397983227 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c0caf50 con 0x7fc49c153e90 2024-01-21T22:24:40.022 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.021+0000 7fc4a1400640 1 --2- 172.21.15.138:0/2397983227 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc49c153e90 0x7fc49c0cc820 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc4940097d0 tx=0x7fc49402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4d4be8022fe5e45 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:40.026 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.025+0000 7fc49b400640 1 -- 172.21.15.138:0/2397983227 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc494031890 con 0x7fc49c153e90 2024-01-21T22:24:40.026 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.025+0000 7fc49b400640 1 -- 172.21.15.138:0/2397983227 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4940319f0 con 0x7fc49c153e90 2024-01-21T22:24:40.026 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.025+0000 7fc49b400640 1 -- 172.21.15.138:0/2397983227 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc494031d80 con 0x7fc49c153e90 2024-01-21T22:24:40.027 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.026+0000 7fc4a1e00640 1 -- 172.21.15.138:0/2397983227 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc49c153e90 msgr2=0x7fc49c0cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:40.027 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.026+0000 7fc4a1e00640 1 --2- 172.21.15.138:0/2397983227 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc49c153e90 0x7fc49c0cc820 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc4940097d0 tx=0x7fc49402b930 comp rx=0 tx=0).stop 2024-01-21T22:24:40.027 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.026+0000 7fc4a1e00640 1 -- 172.21.15.138:0/2397983227 shutdown_connections 2024-01-21T22:24:40.027 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.026+0000 7fc4a1e00640 1 --2- 172.21.15.138:0/2397983227 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc49c153e90 0x7fc49c0cc820 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:40.027 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.026+0000 7fc4a1e00640 1 -- 172.21.15.138:0/2397983227 >> 172.21.15.138:0/2397983227 conn(0x7fc49c06a630 msgr2=0x7fc49c06aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:40.027 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.026+0000 7fc4a1e00640 1 -- 172.21.15.138:0/2397983227 shutdown_connections 2024-01-21T22:24:40.027 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.026+0000 7fc4a1e00640 1 -- 172.21.15.138:0/2397983227 wait complete. 2024-01-21T22:24:40.029 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.028+0000 7fc4a1e00640 1 Processor -- start 2024-01-21T22:24:40.029 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.028+0000 7fc4a1e00640 1 -- start start 2024-01-21T22:24:40.031 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.029+0000 7fc4a1e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc49c070ec0 0x7fc49c1f3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:40.031 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.029+0000 7fc4a1e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc49c0712a0 0x7fc49c14ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:40.031 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.029+0000 7fc4a1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc49c153e90 0x7fc49c0b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:40.031 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.029+0000 7fc4a1e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c1478e0 con 0x7fc49c153e90 2024-01-21T22:24:40.031 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.029+0000 7fc4a1e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c1f3db0 con 0x7fc49c070ec0 2024-01-21T22:24:40.031 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.029+0000 7fc4a1e00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c15afe0 con 0x7fc49c0712a0 2024-01-21T22:24:40.031 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.029+0000 7fc4a1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc49c153e90 0x7fc49c0b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:40.031 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.029+0000 7fc49be00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc49c0712a0 0x7fc49c14ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:40.031 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.029+0000 7fc4a0a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc49c070ec0 0x7fc49c1f3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:40.031 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.030+0000 7fc49be00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc49c0712a0 0x7fc49c14ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:58850/0 (socket says 172.21.15.138:58850) 2024-01-21T22:24:40.031 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.030+0000 7fc49be00640 1 -- 172.21.15.138:0/2742650144 learned_addr learned my addr 172.21.15.138:0/2742650144 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:24:40.034 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.033+0000 7fc4a1400640 1 -- 172.21.15.138:0/2742650144 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc49c0712a0 msgr2=0x7fc49c14ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:40.034 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.033+0000 7fc4a1400640 1 --2- 172.21.15.138:0/2742650144 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc49c0712a0 0x7fc49c14ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:40.035 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.033+0000 7fc4a1400640 1 -- 172.21.15.138:0/2742650144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc49c070ec0 msgr2=0x7fc49c1f3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:40.035 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.033+0000 7fc4a1400640 1 --2- 172.21.15.138:0/2742650144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc49c070ec0 0x7fc49c1f3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:40.035 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.033+0000 7fc4a1400640 1 -- 172.21.15.138:0/2742650144 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc494009400 con 0x7fc49c153e90 2024-01-21T22:24:40.037 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.036+0000 7fc4a1400640 1 --2- 172.21.15.138:0/2742650144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc49c153e90 0x7fc49c0b2820 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc494033000 tx=0x7fc4940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:40.041 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.040+0000 7fc498c00640 1 -- 172.21.15.138:0/2742650144 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4940443f0 con 0x7fc49c153e90 2024-01-21T22:24:40.042 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.040+0000 7fc498c00640 1 -- 172.21.15.138:0/2742650144 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc494044590 con 0x7fc49c153e90 2024-01-21T22:24:40.042 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.040+0000 7fc498c00640 1 -- 172.21.15.138:0/2742650144 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc494044940 con 0x7fc49c153e90 2024-01-21T22:24:40.042 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.040+0000 7fc4a1e00640 1 -- 172.21.15.138:0/2742650144 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49c15b2c0 con 0x7fc49c153e90 2024-01-21T22:24:40.042 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.040+0000 7fc4a1e00640 1 -- 172.21.15.138:0/2742650144 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49c1f3070 con 0x7fc49c153e90 2024-01-21T22:24:40.042 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.041+0000 7fc4a1e00640 1 -- 172.21.15.138:0/2742650144 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc49c15b7e0 con 0x7fc49c153e90 2024-01-21T22:24:40.085 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.084+0000 7fc498c00640 1 -- 172.21.15.138:0/2742650144 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 4) v1 ==== 89460+0+0 (secure 0 0 0) 0x7fc4940303e0 con 0x7fc49c153e90 2024-01-21T22:24:40.087 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.086+0000 7fc498c00640 1 --2- 172.21.15.138:0/2742650144 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fc47406b2d0 0x7fc47406d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:40.088 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.087+0000 7fc4a0a00640 1 --2- 172.21.15.138:0/2742650144 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fc47406b2d0 0x7fc47406d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:40.089 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.088+0000 7fc4a0a00640 1 --2- 172.21.15.138:0/2742650144 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fc47406b2d0 0x7fc47406d790 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc48c001f10 tx=0x7fc48c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:40.091 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.090+0000 7fc498c00640 1 -- 172.21.15.138:0/2742650144 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fc49407b030 con 0x7fc49c153e90 2024-01-21T22:24:40.449 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.448+0000 7fc498c00640 1 -- 172.21.15.138:0/2742650144 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fc494077370 con 0x7fc49c153e90 2024-01-21T22:24:40.651 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.650+0000 7fc4a1e00640 1 -- 172.21.15.138:0/2742650144 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc49c1f2ed0 con 0x7fc49c153e90 2024-01-21T22:24:40.747 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.746+0000 7fc498c00640 1 -- 172.21.15.138:0/2742650144 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fc49c1f2ed0 con 0x7fc49c153e90 2024-01-21T22:24:40.748 INFO:teuthology.misc.health.smithi138.stdout: 2024-01-21T22:24:40.748 INFO:teuthology.misc.health.smithi138.stdout:{"epoch":10,"fsid":"d488d446-149f-48d0-89ca-b2830ae72846","created":"2024-01-21T22:24:28.581932+0000","modified":"2024-01-21T22:24:37.672807+0000","last_up_change":"0.000000","last_in_change":"2024-01-21T22:24:37.672807+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d812b2d6-c8a3-4015-bfdb-5766b25eae5a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d8d875df-c840-46cc-b6c0-a847f9c93999","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4bd94373-76ca-47a6-870b-16d96d7c9e81","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"07267087-d4d2-4e88-ab9b-bb87f6dd061a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b47ef758-d783-4168-a9a5-11db3de4ca12","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0ea5d43e-61d8-4823-815f-9e42c62fae97","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"59ffac36-e2e0-4693-ac82-ad5eccf6224d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4f892c7d-32a5-4960-bf88-4ed7c863cd0a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-21T22:24:40.751 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.749+0000 7fc4a1e00640 1 -- 172.21.15.138:0/2742650144 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fc47406b2d0 msgr2=0x7fc47406d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:40.751 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.750+0000 7fc4a1e00640 1 --2- 172.21.15.138:0/2742650144 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fc47406b2d0 0x7fc47406d790 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc48c001f10 tx=0x7fc48c009040 comp rx=0 tx=0).stop 2024-01-21T22:24:40.751 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.750+0000 7fc4a1e00640 1 -- 172.21.15.138:0/2742650144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc49c153e90 msgr2=0x7fc49c0b2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:40.751 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.750+0000 7fc4a1e00640 1 --2- 172.21.15.138:0/2742650144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc49c153e90 0x7fc49c0b2820 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc494033000 tx=0x7fc4940046c0 comp rx=0 tx=0).stop 2024-01-21T22:24:40.751 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.750+0000 7fc4a1e00640 1 -- 172.21.15.138:0/2742650144 shutdown_connections 2024-01-21T22:24:40.751 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.750+0000 7fc4a1e00640 1 --2- 172.21.15.138:0/2742650144 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fc47406b2d0 0x7fc47406d790 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:40.751 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.750+0000 7fc4a1e00640 1 --2- 172.21.15.138:0/2742650144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc49c153e90 0x7fc49c0b2820 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:40.751 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.750+0000 7fc4a1e00640 1 --2- 172.21.15.138:0/2742650144 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc49c0712a0 0x7fc49c14ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:40.751 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.750+0000 7fc4a1e00640 1 --2- 172.21.15.138:0/2742650144 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc49c070ec0 0x7fc49c1f3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:40.752 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.750+0000 7fc4a1e00640 1 -- 172.21.15.138:0/2742650144 >> 172.21.15.138:0/2742650144 conn(0x7fc49c06a630 msgr2=0x7fc49c0c4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:40.752 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.750+0000 7fc4a1e00640 1 -- 172.21.15.138:0/2742650144 shutdown_connections 2024-01-21T22:24:40.752 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:40.750+0000 7fc4a1e00640 1 -- 172.21.15.138:0/2742650144 wait complete. 2024-01-21T22:24:40.766 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-01-21T22:24:42.127 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-01-21T22:24:42.126+0000 7f078436f640 -1 osd.6 0 log_to_monitors true 2024-01-21T22:24:42.162 INFO:tasks.ceph.osd.7.smithi152.stderr:2024-01-21T22:24:42.161+0000 7fea30437640 -1 osd.7 0 log_to_monitors true 2024-01-21T22:24:42.172 INFO:tasks.ceph.osd.5.smithi152.stderr:2024-01-21T22:24:42.171+0000 7f4b1dcf9640 -1 osd.5 0 log_to_monitors true 2024-01-21T22:24:42.213 INFO:tasks.ceph.osd.4.smithi152.stderr:2024-01-21T22:24:42.212+0000 7f286db38640 -1 osd.4 0 log_to_monitors true 2024-01-21T22:24:42.328 INFO:tasks.ceph.osd.3.smithi138.stderr:2024-01-21T22:24:42.327+0000 7f5d8220c640 -1 osd.3 0 log_to_monitors true 2024-01-21T22:24:42.356 INFO:tasks.ceph.osd.1.smithi138.stderr:2024-01-21T22:24:42.355+0000 7fe1c7e10640 -1 osd.1 0 log_to_monitors true 2024-01-21T22:24:42.395 INFO:tasks.ceph.osd.0.smithi138.stderr:2024-01-21T22:24:42.394+0000 7f7a30d93640 -1 osd.0 0 log_to_monitors true 2024-01-21T22:24:42.578 INFO:tasks.ceph.osd.2.smithi138.stderr:2024-01-21T22:24:42.577+0000 7f3a5d39a640 -1 osd.2 0 log_to_monitors true 2024-01-21T22:24:46.025 INFO:tasks.ceph.osd.3.smithi138.stderr:2024-01-21T22:24:46.023+0000 7f5d7e000640 -1 osd.3 0 waiting for initial osdmap 2024-01-21T22:24:46.060 INFO:tasks.ceph.osd.1.smithi138.stderr:2024-01-21T22:24:46.059+0000 7fe1c3c00640 -1 osd.1 0 waiting for initial osdmap 2024-01-21T22:24:46.063 INFO:tasks.ceph.osd.3.smithi138.stderr:2024-01-21T22:24:46.062+0000 7f5d72c00640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-21T22:24:46.066 INFO:tasks.ceph.osd.0.smithi138.stderr:2024-01-21T22:24:46.065+0000 7f7a2d000640 -1 osd.0 0 waiting for initial osdmap 2024-01-21T22:24:46.110 INFO:tasks.ceph.osd.1.smithi138.stderr:2024-01-21T22:24:46.109+0000 7fe1b8800640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-21T22:24:46.129 INFO:tasks.ceph.osd.5.smithi152.stderr:2024-01-21T22:24:46.128+0000 7f4b19466640 -1 osd.5 0 waiting for initial osdmap 2024-01-21T22:24:46.136 INFO:tasks.ceph.osd.0.smithi138.stderr:2024-01-21T22:24:46.135+0000 7f7a21c00640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-21T22:24:46.152 INFO:tasks.ceph.osd.4.smithi152.stderr:2024-01-21T22:24:46.151+0000 7f28692a5640 -1 osd.4 0 waiting for initial osdmap 2024-01-21T22:24:46.158 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-01-21T22:24:46.157+0000 7f077fadc640 -1 osd.6 0 waiting for initial osdmap 2024-01-21T22:24:46.163 INFO:tasks.ceph.osd.7.smithi152.stderr:2024-01-21T22:24:46.162+0000 7fea2d3dc640 -1 osd.7 0 waiting for initial osdmap 2024-01-21T22:24:46.179 INFO:tasks.ceph.osd.4.smithi152.stderr:2024-01-21T22:24:46.176+0000 7f28618de640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-21T22:24:46.179 INFO:tasks.ceph.osd.5.smithi152.stderr:2024-01-21T22:24:46.176+0000 7f4b11a9f640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-21T22:24:46.180 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-01-21T22:24:46.176+0000 7f0778115640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-21T22:24:46.181 INFO:tasks.ceph.osd.7.smithi152.stderr:2024-01-21T22:24:46.180+0000 7fea241dd640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-21T22:24:46.594 INFO:tasks.ceph.osd.2.smithi138.stderr:2024-01-21T22:24:46.593+0000 7f3a59600640 -1 osd.2 0 waiting for initial osdmap 2024-01-21T22:24:46.603 INFO:tasks.ceph.osd.2.smithi138.stderr:2024-01-21T22:24:46.602+0000 7f3a4e200640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-01-21T22:24:47.572 DEBUG:teuthology.orchestra.run.smithi138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-01-21T22:24:47.745 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.744+0000 7fa3b1400640 1 Processor -- start 2024-01-21T22:24:47.745 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.744+0000 7fa3b1400640 1 -- start start 2024-01-21T22:24:47.746 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.745+0000 7fa3b1400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa3ac071360 0x7fa3ac15da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:47.746 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.745+0000 7fa3b1400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa3ac159c60 0x7fa3ac15a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:47.746 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.745+0000 7fa3b1400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3ac15dfd0 con 0x7fa3ac070f80 2024-01-21T22:24:47.746 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.745+0000 7fa3b1400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3ac0c6080 con 0x7fa3ac159c60 2024-01-21T22:24:47.746 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.745+0000 7fa3b1400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac15a630 con 0x7fa3ac071360 2024-01-21T22:24:47.747 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.746+0000 7fa3ab200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa3ac159c60 0x7fa3ac15a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:47.747 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.746+0000 7fa3aa800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fa3ac070f80 0x7fa3ac0cb0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:34874/0 (socket says 172.21.15.138:34874) 2024-01-21T22:24:47.747 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.746+0000 7fa3aa800640 1 -- 172.21.15.138:0/2508586534 learned_addr learned my addr 172.21.15.138:0/2508586534 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:24:47.750 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.749+0000 7fa3ab200640 1 -- 172.21.15.138:0/2508586534 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa3ac071360 msgr2=0x7fa3ac15da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:47.750 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.749+0000 7fa3ab200640 1 --2- 172.21.15.138:0/2508586534 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa3ac071360 0x7fa3ac15da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:47.750 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.749+0000 7fa3ab200640 1 -- 172.21.15.138:0/2508586534 >> v1:172.21.15.152:6789/0 conn(0x7fa3ac070f80 legacy=0x7fa3ac0cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:47.750 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.749+0000 7fa3ab200640 1 -- 172.21.15.138:0/2508586534 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac0c47f0 con 0x7fa3ac159c60 2024-01-21T22:24:47.754 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.753+0000 7fa3ab200640 1 --2- 172.21.15.138:0/2508586534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3ac159c60 0x7fa3ac15a0f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40097a0 tx=0x7fa3a402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d271e7c61e8ab996 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:47.758 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.757+0000 7fa3a9e00640 1 -- 172.21.15.138:0/2508586534 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4030610 con 0x7fa3ac159c60 2024-01-21T22:24:47.759 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.757+0000 7fa3a9e00640 1 -- 172.21.15.138:0/2508586534 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a4030770 con 0x7fa3ac159c60 2024-01-21T22:24:47.759 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.757+0000 7fa3a9e00640 1 -- 172.21.15.138:0/2508586534 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4030b00 con 0x7fa3ac159c60 2024-01-21T22:24:47.759 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.758+0000 7fa3b1400640 1 -- 172.21.15.138:0/2508586534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3ac159c60 msgr2=0x7fa3ac15a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:47.760 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.758+0000 7fa3b1400640 1 --2- 172.21.15.138:0/2508586534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3ac159c60 0x7fa3ac15a0f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40097a0 tx=0x7fa3a402b960 comp rx=0 tx=0).stop 2024-01-21T22:24:47.760 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.758+0000 7fa3b1400640 1 -- 172.21.15.138:0/2508586534 shutdown_connections 2024-01-21T22:24:47.760 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.758+0000 7fa3b1400640 1 --2- 172.21.15.138:0/2508586534 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa3ac071360 0x7fa3ac15da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:47.760 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.758+0000 7fa3b1400640 1 --2- 172.21.15.138:0/2508586534 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3ac159c60 0x7fa3ac15a0f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:47.760 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.758+0000 7fa3b1400640 1 -- 172.21.15.138:0/2508586534 >> 172.21.15.138:0/2508586534 conn(0x7fa3ac072180 msgr2=0x7fa3ac072590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:47.760 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.758+0000 7fa3b1400640 1 -- 172.21.15.138:0/2508586534 shutdown_connections 2024-01-21T22:24:47.760 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.758+0000 7fa3b1400640 1 -- 172.21.15.138:0/2508586534 wait complete. 2024-01-21T22:24:47.761 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.760+0000 7fa3b1400640 1 Processor -- start 2024-01-21T22:24:47.761 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.760+0000 7fa3b1400640 1 -- start start 2024-01-21T22:24:47.762 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.761+0000 7fa3b1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3ac070f80 0x7fa3ac14c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:47.762 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.761+0000 7fa3b1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3ac071360 0x7fa3ac15c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:47.763 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.761+0000 7fa3b1400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa3ac159c60 0x7fa3ac15c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:47.763 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.761+0000 7fa3b1400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac0c8f50 con 0x7fa3ac071360 2024-01-21T22:24:47.763 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.761+0000 7fa3b1400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac1484a0 con 0x7fa3ac070f80 2024-01-21T22:24:47.763 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.761+0000 7fa3b1400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ac15bf20 con 0x7fa3ac159c60 2024-01-21T22:24:47.763 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.762+0000 7fa3aa800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3ac070f80 0x7fa3ac14c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:47.763 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.762+0000 7fa3aa800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3ac070f80 0x7fa3ac14c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:58626/0 (socket says 172.21.15.138:58626) 2024-01-21T22:24:47.763 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.762+0000 7fa3aa800640 1 -- 172.21.15.138:0/2646449546 learned_addr learned my addr 172.21.15.138:0/2646449546 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:24:47.766 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.765+0000 7fa3aa800640 1 -- 172.21.15.138:0/2646449546 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa3ac159c60 msgr2=0x7fa3ac15c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:47.767 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.765+0000 7fa3aa800640 1 --2- 172.21.15.138:0/2646449546 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa3ac159c60 0x7fa3ac15c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:47.767 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.765+0000 7fa3aa800640 1 -- 172.21.15.138:0/2646449546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3ac071360 msgr2=0x7fa3ac15c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:47.767 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.765+0000 7fa3aa800640 1 --2- 172.21.15.138:0/2646449546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3ac071360 0x7fa3ac15c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:47.767 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.765+0000 7fa3aa800640 1 -- 172.21.15.138:0/2646449546 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a4009400 con 0x7fa3ac070f80 2024-01-21T22:24:47.769 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.768+0000 7fa3aa800640 1 --2- 172.21.15.138:0/2646449546 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3ac070f80 0x7fa3ac14c870 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00cba0 tx=0x7fa39c00a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:47.778 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.777+0000 7fa3a3400640 1 -- 172.21.15.138:0/2646449546 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c00ad40 con 0x7fa3ac070f80 2024-01-21T22:24:47.778 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.777+0000 7fa3a3400640 1 -- 172.21.15.138:0/2646449546 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa39c012400 con 0x7fa3ac070f80 2024-01-21T22:24:47.778 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.777+0000 7fa3a3400640 1 -- 172.21.15.138:0/2646449546 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa39c00aea0 con 0x7fa3ac070f80 2024-01-21T22:24:47.778 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.777+0000 7fa3b1400640 1 -- 172.21.15.138:0/2646449546 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ac14aa70 con 0x7fa3ac070f80 2024-01-21T22:24:47.779 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.777+0000 7fa3b1400640 1 -- 172.21.15.138:0/2646449546 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ac15d970 con 0x7fa3ac070f80 2024-01-21T22:24:47.779 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:47.778+0000 7fa3b1400640 1 -- 172.21.15.138:0/2646449546 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3ac15c120 con 0x7fa3ac070f80 2024-01-21T22:24:48.047 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:48.045+0000 7fa3a3400640 1 -- 172.21.15.138:0/2646449546 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 89460+0+0 (secure 0 0 0) 0x7fa39c007070 con 0x7fa3ac070f80 2024-01-21T22:24:48.049 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:48.048+0000 7fa3a3400640 1 --2- 172.21.15.138:0/2646449546 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fa38406b2f0 0x7fa38406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:48.049 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:48.048+0000 7fa3a3400640 1 -- 172.21.15.138:0/2646449546 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fa39c0897b0 con 0x7fa3ac070f80 2024-01-21T22:24:48.049 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:48.048+0000 7fa3a1c00640 1 --2- 172.21.15.138:0/2646449546 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fa38406b2f0 0x7fa38406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:48.050 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:48.050+0000 7fa3a1c00640 1 --2- 172.21.15.138:0/2646449546 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fa38406b2f0 0x7fa38406d7b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0b5720 tx=0x7fa394006c90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:48.058 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:48.057+0000 7fa3a3400640 1 -- 172.21.15.138:0/2646449546 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa39c054b80 con 0x7fa3ac070f80 2024-01-21T22:24:48.289 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:48.288+0000 7fa3b1400640 1 -- 172.21.15.138:0/2646449546 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3ac15c0e0 con 0x7fa3ac070f80 2024-01-21T22:24:48.344 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:48.342+0000 7fa3a3400640 1 -- 172.21.15.138:0/2646449546 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10921 (secure 0 0 0) 0x7fa39c021090 con 0x7fa3ac070f80 2024-01-21T22:24:48.344 INFO:teuthology.misc.health.smithi138.stdout: 2024-01-21T22:24:48.344 INFO:teuthology.misc.health.smithi138.stdout:{"epoch":14,"fsid":"d488d446-149f-48d0-89ca-b2830ae72846","created":"2024-01-21T22:24:28.581932+0000","modified":"2024-01-21T22:24:47.145799+0000","last_up_change":"2024-01-21T22:24:47.145799+0000","last_in_change":"2024-01-21T22:24:37.672807+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d812b2d6-c8a3-4015-bfdb-5766b25eae5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6825","nonce":2849203277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6827","nonce":2849203277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6831","nonce":2849203277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6829","nonce":2849203277}]},"public_addr":"172.21.15.138:6825/2849203277","cluster_addr":"172.21.15.138:6827/2849203277","heartbeat_back_addr":"172.21.15.138:6831/2849203277","heartbeat_front_addr":"172.21.15.138:6829/2849203277","state":["exists","up"]},{"osd":1,"uuid":"d8d875df-c840-46cc-b6c0-a847f9c93999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6817","nonce":174521731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6819","nonce":174521731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6823","nonce":174521731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6821","nonce":174521731}]},"public_addr":"172.21.15.138:6817/174521731","cluster_addr":"172.21.15.138:6819/174521731","heartbeat_back_addr":"172.21.15.138:6823/174521731","heartbeat_front_addr":"172.21.15.138:6821/174521731","state":["exists","up"]},{"osd":2,"uuid":"4bd94373-76ca-47a6-870b-16d96d7c9e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6801","nonce":1365838748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6803","nonce":1365838748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6809","nonce":1365838748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6805","nonce":1365838748}]},"public_addr":"172.21.15.138:6801/1365838748","cluster_addr":"172.21.15.138:6803/1365838748","heartbeat_back_addr":"172.21.15.138:6809/1365838748","heartbeat_front_addr":"172.21.15.138:6805/1365838748","state":["exists","up"]},{"osd":3,"uuid":"07267087-d4d2-4e88-ab9b-bb87f6dd061a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6808","nonce":1841873885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6811","nonce":1841873885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6815","nonce":1841873885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6813","nonce":1841873885}]},"public_addr":"172.21.15.138:6808/1841873885","cluster_addr":"172.21.15.138:6811/1841873885","heartbeat_back_addr":"172.21.15.138:6815/1841873885","heartbeat_front_addr":"172.21.15.138:6813/1841873885","state":["exists","up"]},{"osd":4,"uuid":"b47ef758-d783-4168-a9a5-11db3de4ca12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6827","nonce":3960812298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6829","nonce":3960812298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6833","nonce":3960812298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6831","nonce":3960812298}]},"public_addr":"172.21.15.152:6827/3960812298","cluster_addr":"172.21.15.152:6829/3960812298","heartbeat_back_addr":"172.21.15.152:6833/3960812298","heartbeat_front_addr":"172.21.15.152:6831/3960812298","state":["exists","up"]},{"osd":5,"uuid":"0ea5d43e-61d8-4823-815f-9e42c62fae97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6819","nonce":4095234314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6821","nonce":4095234314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6825","nonce":4095234314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6823","nonce":4095234314}]},"public_addr":"172.21.15.152:6819/4095234314","cluster_addr":"172.21.15.152:6821/4095234314","heartbeat_back_addr":"172.21.15.152:6825/4095234314","heartbeat_front_addr":"172.21.15.152:6823/4095234314","state":["exists","up"]},{"osd":6,"uuid":"59ffac36-e2e0-4693-ac82-ad5eccf6224d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6803","nonce":223848127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6805","nonce":223848127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6809","nonce":223848127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6807","nonce":223848127}]},"public_addr":"172.21.15.152:6803/223848127","cluster_addr":"172.21.15.152:6805/223848127","heartbeat_back_addr":"172.21.15.152:6809/223848127","heartbeat_front_addr":"172.21.15.152:6807/223848127","state":["exists","up"]},{"osd":7,"uuid":"4f892c7d-32a5-4960-bf88-4ed7c863cd0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6811","nonce":52509095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6813","nonce":52509095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6817","nonce":52509095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6815","nonce":52509095}]},"public_addr":"172.21.15.152:6811/52509095","cluster_addr":"172.21.15.152:6813/52509095","heartbeat_back_addr":"172.21.15.152:6817/52509095","heartbeat_front_addr":"172.21.15.152:6815/52509095","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-21T22:24:48.347 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:48.346+0000 7fa3b1400640 1 -- 172.21.15.138:0/2646449546 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fa38406b2f0 msgr2=0x7fa38406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:48.347 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:48.346+0000 7fa3b1400640 1 --2- 172.21.15.138:0/2646449546 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fa38406b2f0 0x7fa38406d7b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac0b5720 tx=0x7fa394006c90 comp rx=0 tx=0).stop 2024-01-21T22:24:48.347 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:48.346+0000 7fa3b1400640 1 -- 172.21.15.138:0/2646449546 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3ac070f80 msgr2=0x7fa3ac14c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:48.347 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:48.346+0000 7fa3b1400640 1 --2- 172.21.15.138:0/2646449546 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3ac070f80 0x7fa3ac14c870 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00cba0 tx=0x7fa39c00a430 comp rx=0 tx=0).stop 2024-01-21T22:24:48.347 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:48.346+0000 7fa3b1400640 1 -- 172.21.15.138:0/2646449546 shutdown_connections 2024-01-21T22:24:48.347 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:48.346+0000 7fa3b1400640 1 --2- 172.21.15.138:0/2646449546 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fa38406b2f0 0x7fa38406d7b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:48.347 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:48.346+0000 7fa3b1400640 1 --2- 172.21.15.138:0/2646449546 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa3ac159c60 0x7fa3ac15c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:48.348 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:48.346+0000 7fa3b1400640 1 --2- 172.21.15.138:0/2646449546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3ac071360 0x7fa3ac15c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:48.348 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:48.346+0000 7fa3b1400640 1 --2- 172.21.15.138:0/2646449546 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa3ac070f80 0x7fa3ac14c870 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:48.348 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:48.346+0000 7fa3b1400640 1 -- 172.21.15.138:0/2646449546 >> 172.21.15.138:0/2646449546 conn(0x7fa3ac072180 msgr2=0x7fa3ac072560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:48.348 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:48.346+0000 7fa3b1400640 1 -- 172.21.15.138:0/2646449546 shutdown_connections 2024-01-21T22:24:48.348 INFO:teuthology.misc.health.smithi138.stderr:2024-01-21T22:24:48.346+0000 7fa3b1400640 1 -- 172.21.15.138:0/2646449546 wait complete. 2024-01-21T22:24:48.364 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-01-21T22:24:48.364 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-01-21T22:24:48.364 DEBUG:tasks.ceph_manager:running mds.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.a.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-01-21T22:24:48.364 INFO:tasks.ceph.mds.a:Restarting daemon 2024-01-21T22:24:48.364 DEBUG:teuthology.orchestra.run.smithi138:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i a 2024-01-21T22:24:48.373 INFO:tasks.ceph.mds.a:Started 2024-01-21T22:24:48.373 DEBUG:tasks.ceph_manager:running mds.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.c.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-01-21T22:24:48.374 INFO:tasks.ceph.mds.c:Restarting daemon 2024-01-21T22:24:48.374 DEBUG:teuthology.orchestra.run.smithi138:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i c 2024-01-21T22:24:48.383 INFO:tasks.ceph.mds.c:Started 2024-01-21T22:24:48.383 DEBUG:tasks.ceph_manager:running mds.e under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.e.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-01-21T22:24:48.383 INFO:tasks.ceph.mds.e:Restarting daemon 2024-01-21T22:24:48.383 DEBUG:teuthology.orchestra.run.smithi138:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.e.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i e 2024-01-21T22:24:48.390 INFO:tasks.ceph.mds.e:Started 2024-01-21T22:24:48.390 DEBUG:tasks.ceph_manager:running mds.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.b.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-01-21T22:24:48.390 INFO:tasks.ceph.mds.b:Restarting daemon 2024-01-21T22:24:48.391 DEBUG:teuthology.orchestra.run.smithi152:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i b 2024-01-21T22:24:48.393 INFO:tasks.ceph.mds.b:Started 2024-01-21T22:24:48.393 DEBUG:tasks.ceph_manager:running mds.d under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.d.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-01-21T22:24:48.393 INFO:tasks.ceph.mds.d:Restarting daemon 2024-01-21T22:24:48.394 DEBUG:teuthology.orchestra.run.smithi152:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.d.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i d 2024-01-21T22:24:48.395 INFO:tasks.ceph.mds.d:Started 2024-01-21T22:24:48.395 DEBUG:tasks.ceph_manager:running mds.f under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mds.f.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-01-21T22:24:48.396 INFO:tasks.ceph.mds.f:Restarting daemon 2024-01-21T22:24:48.396 DEBUG:teuthology.orchestra.run.smithi152:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.f.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i f 2024-01-21T22:24:48.398 INFO:tasks.ceph.mds.f:Started 2024-01-21T22:24:48.398 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-01-21T22:24:48.398 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-21T22:24:48.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.598+0000 7fd1d7000640 1 Processor -- start 2024-01-21T22:24:48.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.598+0000 7fd1d7000640 1 -- start start 2024-01-21T22:24:48.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.599+0000 7fd1d7000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd1d0151a30 0x7fd1d00c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:48.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.599+0000 7fd1d7000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd1d0151e10 0x7fd1d015ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:48.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.599+0000 7fd1d7000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1d01583d0 con 0x7fd1d015b3a0 2024-01-21T22:24:48.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.599+0000 7fd1d7000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1d006c7b0 con 0x7fd1d0151e10 2024-01-21T22:24:48.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.599+0000 7fd1d7000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1d0157890 con 0x7fd1d0151a30 2024-01-21T22:24:48.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.600+0000 7fd1cf400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd1d0151e10 0x7fd1d015ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:48.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.600+0000 7fd1cfe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd1d0151a30 0x7fd1d00c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:48.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.600+0000 7fd1d5000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd1d015b3a0 0x7fd1d00c47c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:59898/0 (socket says 172.21.15.138:59898) 2024-01-21T22:24:48.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.600+0000 7fd1d5000640 1 -- 172.21.15.138:0/162957185 learned_addr learned my addr 172.21.15.138:0/162957185 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:24:48.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.600+0000 7fd1cf400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd1d0151e10 0x7fd1d015ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42194/0 (socket says 172.21.15.138:42194) 2024-01-21T22:24:48.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.603+0000 7fd1cf400640 1 -- 172.21.15.138:0/162957185 >> v2:172.21.15.152:3300/0 conn(0x7fd1d0151a30 msgr2=0x7fd1d00c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:48.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.603+0000 7fd1cf400640 1 --2- 172.21.15.138:0/162957185 >> v2:172.21.15.152:3300/0 conn(0x7fd1d0151a30 0x7fd1d00c5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:48.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.603+0000 7fd1cf400640 1 -- 172.21.15.138:0/162957185 >> v1:172.21.15.152:6789/0 conn(0x7fd1d015b3a0 legacy=0x7fd1d00c47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:48.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.603+0000 7fd1cf400640 1 -- 172.21.15.138:0/162957185 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d00c5ff0 con 0x7fd1d0151e10 2024-01-21T22:24:48.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.606+0000 7fd1cf400640 1 --2- 172.21.15.138:0/162957185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1d0151e10 0x7fd1d015ae60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd1b80097d0 tx=0x7fd1b802b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a0782c25e3fbe7 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:48.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.610+0000 7fd1cea00640 1 -- 172.21.15.138:0/162957185 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8030800 con 0x7fd1d0151e10 2024-01-21T22:24:48.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.610+0000 7fd1cea00640 1 -- 172.21.15.138:0/162957185 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b8030960 con 0x7fd1d0151e10 2024-01-21T22:24:48.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.610+0000 7fd1cea00640 1 -- 172.21.15.138:0/162957185 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b8030cf0 con 0x7fd1d0151e10 2024-01-21T22:24:48.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.611+0000 7fd1d7000640 1 -- 172.21.15.138:0/162957185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1d0151e10 msgr2=0x7fd1d015ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:48.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.611+0000 7fd1d7000640 1 --2- 172.21.15.138:0/162957185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1d0151e10 0x7fd1d015ae60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd1b80097d0 tx=0x7fd1b802b940 comp rx=0 tx=0).stop 2024-01-21T22:24:48.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.611+0000 7fd1d7000640 1 -- 172.21.15.138:0/162957185 shutdown_connections 2024-01-21T22:24:48.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.612+0000 7fd1d7000640 1 --2- 172.21.15.138:0/162957185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1d0151e10 0x7fd1d015ae60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:48.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.612+0000 7fd1d7000640 1 --2- 172.21.15.138:0/162957185 >> v2:172.21.15.152:3300/0 conn(0x7fd1d0151a30 0x7fd1d00c5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:48.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.612+0000 7fd1d7000640 1 -- 172.21.15.138:0/162957185 >> 172.21.15.138:0/162957185 conn(0x7fd1d00cb0b0 msgr2=0x7fd1d00cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:48.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.612+0000 7fd1d7000640 1 -- 172.21.15.138:0/162957185 shutdown_connections 2024-01-21T22:24:48.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.612+0000 7fd1d7000640 1 -- 172.21.15.138:0/162957185 wait complete. 2024-01-21T22:24:48.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.614+0000 7fd1d7000640 1 Processor -- start 2024-01-21T22:24:48.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.615+0000 7fd1d7000640 1 -- start start 2024-01-21T22:24:48.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.616+0000 7fd1d7000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1d0151a30 0x7fd1d0159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:48.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.616+0000 7fd1d7000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1d015b3a0 0x7fd1d0152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:48.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.616+0000 7fd1d7000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1d0154d00 0x7fd1d0143dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:48.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.616+0000 7fd1d7000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d0152de0 con 0x7fd1d0151a30 2024-01-21T22:24:48.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.616+0000 7fd1d7000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d0144e40 con 0x7fd1d0154d00 2024-01-21T22:24:48.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.616+0000 7fd1d7000640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d01456e0 con 0x7fd1d015b3a0 2024-01-21T22:24:48.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.617+0000 7fd1cfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1d0151a30 0x7fd1d0159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:48.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.617+0000 7fd1d5000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1d0154d00 0x7fd1d0143dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:48.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.617+0000 7fd1d5000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1d0154d00 0x7fd1d0143dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:43564/0 (socket says 172.21.15.138:43564) 2024-01-21T22:24:48.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.617+0000 7fd1d5000640 1 -- 172.21.15.138:0/1545903513 learned_addr learned my addr 172.21.15.138:0/1545903513 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:24:48.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.622+0000 7fd1d5000640 1 -- 172.21.15.138:0/1545903513 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1d015b3a0 msgr2=0x7fd1d0152470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:48.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.622+0000 7fd1d5000640 1 --2- 172.21.15.138:0/1545903513 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1d015b3a0 0x7fd1d0152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:48.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.622+0000 7fd1d5000640 1 -- 172.21.15.138:0/1545903513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1d0151a30 msgr2=0x7fd1d0159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:48.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.622+0000 7fd1d5000640 1 --2- 172.21.15.138:0/1545903513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1d0151a30 0x7fd1d0159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:48.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.623+0000 7fd1d5000640 1 -- 172.21.15.138:0/1545903513 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b8009400 con 0x7fd1d0154d00 2024-01-21T22:24:48.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.626+0000 7fd1d5000640 1 --2- 172.21.15.138:0/1545903513 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1d0154d00 0x7fd1d0143dc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800cad0 tx=0x7fd1c800cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:48.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.631+0000 7fd1c7e00640 1 -- 172.21.15.138:0/1545903513 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c800ab70 con 0x7fd1d0154d00 2024-01-21T22:24:48.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.632+0000 7fd1c7e00640 1 -- 172.21.15.138:0/1545903513 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c800acd0 con 0x7fd1d0154d00 2024-01-21T22:24:48.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.632+0000 7fd1c7e00640 1 -- 172.21.15.138:0/1545903513 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8012600 con 0x7fd1d0154d00 2024-01-21T22:24:48.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.632+0000 7fd1d7000640 1 -- 172.21.15.138:0/1545903513 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d0159e80 con 0x7fd1d0154d00 2024-01-21T22:24:48.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.632+0000 7fd1d7000640 1 -- 172.21.15.138:0/1545903513 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d0158e40 con 0x7fd1d0154d00 2024-01-21T22:24:48.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.633+0000 7fd1d7000640 1 -- 172.21.15.138:0/1545903513 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d0158820 con 0x7fd1d0154d00 2024-01-21T22:24:48.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.666+0000 7fd1c7e00640 1 -- 172.21.15.138:0/1545903513 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 89460+0+0 (secure 0 0 0) 0x7fd1c8007070 con 0x7fd1d0154d00 2024-01-21T22:24:48.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.668+0000 7fd1c7e00640 1 --2- 172.21.15.138:0/1545903513 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd1a806b2f0 0x7fd1a806d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:48.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.668+0000 7fd1cfe00640 1 --2- 172.21.15.138:0/1545903513 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd1a806b2f0 0x7fd1a806d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:48.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.669+0000 7fd1cfe00640 1 --2- 172.21.15.138:0/1545903513 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd1a806b2f0 0x7fd1a806d7b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0009b40 tx=0x7fd1c0009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:48.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.670+0000 7fd1c7e00640 1 -- 172.21.15.138:0/1545903513 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fd1c8058660 con 0x7fd1d0154d00 2024-01-21T22:24:48.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:48.954+0000 7fd1c7e00640 1 -- 172.21.15.138:0/1545903513 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd1c80549a0 con 0x7fd1d0154d00 2024-01-21T22:24:49.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.154+0000 7fd1d7000640 1 -- 172.21.15.138:0/1545903513 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1d0158820 con 0x7fd1d0154d00 2024-01-21T22:24:49.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.298+0000 7fd1c7e00640 1 -- 172.21.15.138:0/1545903513 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10921 (secure 0 0 0) 0x7fd1d0158820 con 0x7fd1d0154d00 2024-01-21T22:24:49.301 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:24:49.301 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":14,"fsid":"d488d446-149f-48d0-89ca-b2830ae72846","created":"2024-01-21T22:24:28.581932+0000","modified":"2024-01-21T22:24:47.145799+0000","last_up_change":"2024-01-21T22:24:47.145799+0000","last_in_change":"2024-01-21T22:24:37.672807+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"d812b2d6-c8a3-4015-bfdb-5766b25eae5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6825","nonce":2849203277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6827","nonce":2849203277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6831","nonce":2849203277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6829","nonce":2849203277}]},"public_addr":"172.21.15.138:6825/2849203277","cluster_addr":"172.21.15.138:6827/2849203277","heartbeat_back_addr":"172.21.15.138:6831/2849203277","heartbeat_front_addr":"172.21.15.138:6829/2849203277","state":["exists","up"]},{"osd":1,"uuid":"d8d875df-c840-46cc-b6c0-a847f9c93999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6817","nonce":174521731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6819","nonce":174521731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6823","nonce":174521731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6821","nonce":174521731}]},"public_addr":"172.21.15.138:6817/174521731","cluster_addr":"172.21.15.138:6819/174521731","heartbeat_back_addr":"172.21.15.138:6823/174521731","heartbeat_front_addr":"172.21.15.138:6821/174521731","state":["exists","up"]},{"osd":2,"uuid":"4bd94373-76ca-47a6-870b-16d96d7c9e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6801","nonce":1365838748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6803","nonce":1365838748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6809","nonce":1365838748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6805","nonce":1365838748}]},"public_addr":"172.21.15.138:6801/1365838748","cluster_addr":"172.21.15.138:6803/1365838748","heartbeat_back_addr":"172.21.15.138:6809/1365838748","heartbeat_front_addr":"172.21.15.138:6805/1365838748","state":["exists","up"]},{"osd":3,"uuid":"07267087-d4d2-4e88-ab9b-bb87f6dd061a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6808","nonce":1841873885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6811","nonce":1841873885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6815","nonce":1841873885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6813","nonce":1841873885}]},"public_addr":"172.21.15.138:6808/1841873885","cluster_addr":"172.21.15.138:6811/1841873885","heartbeat_back_addr":"172.21.15.138:6815/1841873885","heartbeat_front_addr":"172.21.15.138:6813/1841873885","state":["exists","up"]},{"osd":4,"uuid":"b47ef758-d783-4168-a9a5-11db3de4ca12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6827","nonce":3960812298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6829","nonce":3960812298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6833","nonce":3960812298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6831","nonce":3960812298}]},"public_addr":"172.21.15.152:6827/3960812298","cluster_addr":"172.21.15.152:6829/3960812298","heartbeat_back_addr":"172.21.15.152:6833/3960812298","heartbeat_front_addr":"172.21.15.152:6831/3960812298","state":["exists","up"]},{"osd":5,"uuid":"0ea5d43e-61d8-4823-815f-9e42c62fae97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6819","nonce":4095234314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6821","nonce":4095234314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6825","nonce":4095234314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6823","nonce":4095234314}]},"public_addr":"172.21.15.152:6819/4095234314","cluster_addr":"172.21.15.152:6821/4095234314","heartbeat_back_addr":"172.21.15.152:6825/4095234314","heartbeat_front_addr":"172.21.15.152:6823/4095234314","state":["exists","up"]},{"osd":6,"uuid":"59ffac36-e2e0-4693-ac82-ad5eccf6224d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6803","nonce":223848127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6805","nonce":223848127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6809","nonce":223848127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6807","nonce":223848127}]},"public_addr":"172.21.15.152:6803/223848127","cluster_addr":"172.21.15.152:6805/223848127","heartbeat_back_addr":"172.21.15.152:6809/223848127","heartbeat_front_addr":"172.21.15.152:6807/223848127","state":["exists","up"]},{"osd":7,"uuid":"4f892c7d-32a5-4960-bf88-4ed7c863cd0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6811","nonce":52509095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6813","nonce":52509095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6817","nonce":52509095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6815","nonce":52509095}]},"public_addr":"172.21.15.152:6811/52509095","cluster_addr":"172.21.15.152:6813/52509095","heartbeat_back_addr":"172.21.15.152:6817/52509095","heartbeat_front_addr":"172.21.15.152:6815/52509095","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-21T22:24:49.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.301+0000 7fd1d7000640 1 -- 172.21.15.138:0/1545903513 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd1a806b2f0 msgr2=0x7fd1a806d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:49.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.301+0000 7fd1d7000640 1 --2- 172.21.15.138:0/1545903513 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd1a806b2f0 0x7fd1a806d7b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd1c0009b40 tx=0x7fd1c0009190 comp rx=0 tx=0).stop 2024-01-21T22:24:49.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.301+0000 7fd1d7000640 1 -- 172.21.15.138:0/1545903513 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1d0154d00 msgr2=0x7fd1d0143dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:49.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.301+0000 7fd1d7000640 1 --2- 172.21.15.138:0/1545903513 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1d0154d00 0x7fd1d0143dc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800cad0 tx=0x7fd1c800cf90 comp rx=0 tx=0).stop 2024-01-21T22:24:49.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.301+0000 7fd1d7000640 1 -- 172.21.15.138:0/1545903513 shutdown_connections 2024-01-21T22:24:49.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.301+0000 7fd1d7000640 1 --2- 172.21.15.138:0/1545903513 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd1a806b2f0 0x7fd1a806d7b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:49.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.301+0000 7fd1d7000640 1 --2- 172.21.15.138:0/1545903513 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd1d0154d00 0x7fd1d0143dc0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:49.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.301+0000 7fd1d7000640 1 --2- 172.21.15.138:0/1545903513 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd1d015b3a0 0x7fd1d0152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:49.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.301+0000 7fd1d7000640 1 --2- 172.21.15.138:0/1545903513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd1d0151a30 0x7fd1d0159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:49.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.301+0000 7fd1d7000640 1 -- 172.21.15.138:0/1545903513 >> 172.21.15.138:0/1545903513 conn(0x7fd1d00cb0b0 msgr2=0x7fd1d0071000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:49.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.301+0000 7fd1d7000640 1 -- 172.21.15.138:0/1545903513 shutdown_connections 2024-01-21T22:24:49.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.301+0000 7fd1d7000640 1 -- 172.21.15.138:0/1545903513 wait complete. 2024-01-21T22:24:49.321 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-01-21T22:24:49.322 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-21T22:24:49.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.515+0000 7f3714a00640 1 Processor -- start 2024-01-21T22:24:49.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.516+0000 7f3714a00640 1 -- start start 2024-01-21T22:24:49.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.517+0000 7f3714a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f370c153e70 0x7f370c0c7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:49.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.517+0000 7f3714a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f370c0c68e0 0x7f370c0cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:49.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.517+0000 7f3714a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370c0c7270 con 0x7f370c0c9bb0 2024-01-21T22:24:49.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.517+0000 7f3714a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f370c15a550 con 0x7f370c153e70 2024-01-21T22:24:49.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.517+0000 7f3714a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f370c0cb620 con 0x7f370c0c68e0 2024-01-21T22:24:49.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.518+0000 7f3712a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f370c0c68e0 0x7f370c0cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:49.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.518+0000 7f3712000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f370c153e70 0x7f370c0c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:49.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.519+0000 7f3711600640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f370c0c9bb0 0x7f370c0ca8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:59904/0 (socket says 172.21.15.138:59904) 2024-01-21T22:24:49.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.519+0000 7f3711600640 1 -- 172.21.15.138:0/2359452026 learned_addr learned my addr 172.21.15.138:0/2359452026 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:24:49.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.522+0000 7f3712000640 1 -- 172.21.15.138:0/2359452026 >> v2:172.21.15.152:3300/0 conn(0x7f370c0c68e0 msgr2=0x7f370c0cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:49.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.522+0000 7f3712000640 1 --2- 172.21.15.138:0/2359452026 >> v2:172.21.15.152:3300/0 conn(0x7f370c0c68e0 0x7f370c0cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:49.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.522+0000 7f3712000640 1 -- 172.21.15.138:0/2359452026 >> v1:172.21.15.152:6789/0 conn(0x7f370c0c9bb0 legacy=0x7f370c0ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:49.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.522+0000 7f3712000640 1 -- 172.21.15.138:0/2359452026 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c0ca0f0 con 0x7f370c153e70 2024-01-21T22:24:49.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.525+0000 7f3712000640 1 --2- 172.21.15.138:0/2359452026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f370c153e70 0x7f370c0c7700 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f370c06ccf0 tx=0x7f36fc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4289ffaa59b92ffd server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:49.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.530+0000 7f3710c00640 1 -- 172.21.15.138:0/2359452026 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc031830 con 0x7f370c153e70 2024-01-21T22:24:49.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.530+0000 7f3710c00640 1 -- 172.21.15.138:0/2359452026 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc031990 con 0x7f370c153e70 2024-01-21T22:24:49.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.530+0000 7f3710c00640 1 -- 172.21.15.138:0/2359452026 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc031d20 con 0x7f370c153e70 2024-01-21T22:24:49.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.531+0000 7f3714a00640 1 -- 172.21.15.138:0/2359452026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f370c153e70 msgr2=0x7f370c0c7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:49.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.531+0000 7f3714a00640 1 --2- 172.21.15.138:0/2359452026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f370c153e70 0x7f370c0c7700 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f370c06ccf0 tx=0x7f36fc02b8d0 comp rx=0 tx=0).stop 2024-01-21T22:24:49.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.531+0000 7f3714a00640 1 -- 172.21.15.138:0/2359452026 shutdown_connections 2024-01-21T22:24:49.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.531+0000 7f3714a00640 1 --2- 172.21.15.138:0/2359452026 >> v2:172.21.15.152:3300/0 conn(0x7f370c0c68e0 0x7f370c0cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:49.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.531+0000 7f3714a00640 1 --2- 172.21.15.138:0/2359452026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f370c153e70 0x7f370c0c7700 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:49.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.531+0000 7f3714a00640 1 -- 172.21.15.138:0/2359452026 >> 172.21.15.138:0/2359452026 conn(0x7f370c0cf410 msgr2=0x7f370c0cf820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:49.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.531+0000 7f3714a00640 1 -- 172.21.15.138:0/2359452026 shutdown_connections 2024-01-21T22:24:49.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.532+0000 7f3714a00640 1 -- 172.21.15.138:0/2359452026 wait complete. 2024-01-21T22:24:49.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.534+0000 7f3714a00640 1 Processor -- start 2024-01-21T22:24:49.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.534+0000 7f3714a00640 1 -- start start 2024-01-21T22:24:49.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.536+0000 7f3714a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f370c0c68e0 0x7f370c14ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:49.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.536+0000 7f3714a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f370c0c9bb0 0x7f370c1f0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:49.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.536+0000 7f3714a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f370c153e70 0x7f370c151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:49.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.536+0000 7f3714a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c1f3b40 con 0x7f370c153e70 2024-01-21T22:24:49.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.536+0000 7f3714a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c1f0880 con 0x7f370c0c68e0 2024-01-21T22:24:49.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.536+0000 7f3714a00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f370c14f380 con 0x7f370c0c9bb0 2024-01-21T22:24:49.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.537+0000 7f3712000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f370c0c68e0 0x7f370c14ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:49.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.537+0000 7f3711600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f370c0c9bb0 0x7f370c1f0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:49.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.537+0000 7f3712000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f370c0c68e0 0x7f370c14ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:43580/0 (socket says 172.21.15.138:43580) 2024-01-21T22:24:49.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.537+0000 7f3712000640 1 -- 172.21.15.138:0/3813548681 learned_addr learned my addr 172.21.15.138:0/3813548681 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:24:49.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.540+0000 7f3712000640 1 -- 172.21.15.138:0/3813548681 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f370c0c9bb0 msgr2=0x7f370c1f0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:49.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.540+0000 7f3712000640 1 --2- 172.21.15.138:0/3813548681 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f370c0c9bb0 0x7f370c1f0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:49.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.540+0000 7f3712000640 1 -- 172.21.15.138:0/3813548681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f370c153e70 msgr2=0x7f370c151650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:49.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.540+0000 7f3712000640 1 --2- 172.21.15.138:0/3813548681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f370c153e70 0x7f370c151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:49.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.540+0000 7f3712000640 1 -- 172.21.15.138:0/3813548681 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc009400 con 0x7f370c0c68e0 2024-01-21T22:24:49.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.543+0000 7f3712000640 1 --2- 172.21.15.138:0/3813548681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f370c0c68e0 0x7f370c14ec80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f36fc000c00 tx=0x7f36fc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:49.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.545+0000 7f370a000640 1 -- 172.21.15.138:0/3813548681 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc03a8f0 con 0x7f370c0c68e0 2024-01-21T22:24:49.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.545+0000 7f3714a00640 1 -- 172.21.15.138:0/3813548681 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370c1f2f30 con 0x7f370c0c68e0 2024-01-21T22:24:49.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.545+0000 7f3714a00640 1 -- 172.21.15.138:0/3813548681 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f370c1f34f0 con 0x7f370c0c68e0 2024-01-21T22:24:49.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.546+0000 7f3714a00640 1 -- 172.21.15.138:0/3813548681 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f370c0c9880 con 0x7f370c0c68e0 2024-01-21T22:24:49.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.548+0000 7f370a000640 1 -- 172.21.15.138:0/3813548681 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc03aa50 con 0x7f370c0c68e0 2024-01-21T22:24:49.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.548+0000 7f370a000640 1 -- 172.21.15.138:0/3813548681 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc03adc0 con 0x7f370c0c68e0 2024-01-21T22:24:49.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.826+0000 7f370a000640 1 -- 172.21.15.138:0/3813548681 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 89460+0+0 (secure 0 0 0) 0x7f36fc04d450 con 0x7f370c0c68e0 2024-01-21T22:24:49.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.827+0000 7f370a000640 1 --2- 172.21.15.138:0/3813548681 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f36e406b2a0 0x7f36e406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:49.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.828+0000 7f3711600640 1 --2- 172.21.15.138:0/3813548681 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f36e406b2a0 0x7f36e406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:49.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.828+0000 7f3711600640 1 --2- 172.21.15.138:0/3813548681 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f36e406b2a0 0x7f36e406d760 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f370c0cd2d0 tx=0x7f36f4007e80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:49.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.830+0000 7f370a000640 1 -- 172.21.15.138:0/3813548681 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f36fc07b120 con 0x7f370c0c68e0 2024-01-21T22:24:49.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:49.841+0000 7f370a000640 1 -- 172.21.15.138:0/3813548681 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f36fc076d60 con 0x7f370c0c68e0 2024-01-21T22:24:50.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:50.093+0000 7f3714a00640 1 -- 172.21.15.138:0/3813548681 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f370c04eaa0 con 0x7f370c0c68e0 2024-01-21T22:24:50.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:50.115+0000 7f370a000640 1 -- 172.21.15.138:0/3813548681 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f36fc07a380 con 0x7f370c0c68e0 2024-01-21T22:24:50.117 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:24:50.117 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-01-21T22:24:50.117 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 1 2024-01-21T22:24:50.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:50.118+0000 7f3714a00640 1 -- 172.21.15.138:0/3813548681 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f36e406b2a0 msgr2=0x7f36e406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:50.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:50.118+0000 7f3714a00640 1 --2- 172.21.15.138:0/3813548681 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f36e406b2a0 0x7f36e406d760 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f370c0cd2d0 tx=0x7f36f4007e80 comp rx=0 tx=0).stop 2024-01-21T22:24:50.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:50.118+0000 7f3714a00640 1 -- 172.21.15.138:0/3813548681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f370c0c68e0 msgr2=0x7f370c14ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:50.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:50.118+0000 7f3714a00640 1 --2- 172.21.15.138:0/3813548681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f370c0c68e0 0x7f370c14ec80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f36fc000c00 tx=0x7f36fc0046c0 comp rx=0 tx=0).stop 2024-01-21T22:24:50.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:50.118+0000 7f3714a00640 1 -- 172.21.15.138:0/3813548681 shutdown_connections 2024-01-21T22:24:50.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:50.118+0000 7f3714a00640 1 --2- 172.21.15.138:0/3813548681 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f36e406b2a0 0x7f36e406d760 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:50.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:50.118+0000 7f3714a00640 1 --2- 172.21.15.138:0/3813548681 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f370c153e70 0x7f370c151650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:50.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:50.118+0000 7f3714a00640 1 --2- 172.21.15.138:0/3813548681 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f370c0c9bb0 0x7f370c1f0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:50.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:50.118+0000 7f3714a00640 1 --2- 172.21.15.138:0/3813548681 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f370c0c68e0 0x7f370c14ec80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:50.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:50.118+0000 7f3714a00640 1 -- 172.21.15.138:0/3813548681 >> 172.21.15.138:0/3813548681 conn(0x7f370c0cf410 msgr2=0x7f370c06abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:50.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:50.118+0000 7f3714a00640 1 -- 172.21.15.138:0/3813548681 shutdown_connections 2024-01-21T22:24:50.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:50.118+0000 7f3714a00640 1 -- 172.21.15.138:0/3813548681 wait complete. 2024-01-21T22:24:52.140 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-21T22:24:52.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.329+0000 7f4753e00640 1 Processor -- start 2024-01-21T22:24:52.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.329+0000 7f4753e00640 1 -- start start 2024-01-21T22:24:52.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.330+0000 7f4753e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4754153a50 0x7f47540c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:52.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.330+0000 7f4753e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f475406c0a0 0x7f47540cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:52.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.330+0000 7f4753e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47540ca000 con 0x7f47540cc090 2024-01-21T22:24:52.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.330+0000 7f4753e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f475406ad30 con 0x7f4754153a50 2024-01-21T22:24:52.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.330+0000 7f4753e00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47540cb8d0 con 0x7f475406c0a0 2024-01-21T22:24:52.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.331+0000 7f4752000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f475406c0a0 0x7f47540cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:52.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.332+0000 7f4753400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f47540cc090 0x7f475406b1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:37184/0 (socket says 172.21.15.138:37184) 2024-01-21T22:24:52.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.332+0000 7f4753400640 1 -- 172.21.15.138:0/343446525 learned_addr learned my addr 172.21.15.138:0/343446525 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:24:52.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.334+0000 7f4752000640 1 -- 172.21.15.138:0/343446525 >> v2:172.21.15.152:3300/0 conn(0x7f4754153a50 msgr2=0x7f47540c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:52.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.334+0000 7f4752000640 1 --2- 172.21.15.138:0/343446525 >> v2:172.21.15.152:3300/0 conn(0x7f4754153a50 0x7f47540c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:52.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.334+0000 7f4752000640 1 -- 172.21.15.138:0/343446525 >> v1:172.21.15.138:6789/0 conn(0x7f47540cc090 legacy=0x7f475406b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:52.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.334+0000 7f4752000640 1 -- 172.21.15.138:0/343446525 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475415d120 con 0x7f475406c0a0 2024-01-21T22:24:52.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.338+0000 7f4752000640 1 --2- 172.21.15.138:0/343446525 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f475406c0a0 0x7f47540cbb50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f473c0097a0 tx=0x7f473c02b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a284a316d9d85c3 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:52.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.342+0000 7f4751600640 1 -- 172.21.15.138:0/343446525 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c031780 con 0x7f475406c0a0 2024-01-21T22:24:52.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.342+0000 7f4751600640 1 -- 172.21.15.138:0/343446525 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473c0318e0 con 0x7f475406c0a0 2024-01-21T22:24:52.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.342+0000 7f4751600640 1 -- 172.21.15.138:0/343446525 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c031c70 con 0x7f475406c0a0 2024-01-21T22:24:52.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.343+0000 7f4753e00640 1 -- 172.21.15.138:0/343446525 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f475406c0a0 msgr2=0x7f47540cbb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:52.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.343+0000 7f4753e00640 1 --2- 172.21.15.138:0/343446525 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f475406c0a0 0x7f47540cbb50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f473c0097a0 tx=0x7f473c02b820 comp rx=0 tx=0).stop 2024-01-21T22:24:52.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.343+0000 7f4753e00640 1 -- 172.21.15.138:0/343446525 shutdown_connections 2024-01-21T22:24:52.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.343+0000 7f4753e00640 1 --2- 172.21.15.138:0/343446525 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f475406c0a0 0x7f47540cbb50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:52.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.343+0000 7f4753e00640 1 --2- 172.21.15.138:0/343446525 >> v2:172.21.15.152:3300/0 conn(0x7f4754153a50 0x7f47540c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:52.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.343+0000 7f4753e00640 1 -- 172.21.15.138:0/343446525 >> 172.21.15.138:0/343446525 conn(0x7f475406d240 msgr2=0x7f4754066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:52.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.343+0000 7f4753e00640 1 -- 172.21.15.138:0/343446525 shutdown_connections 2024-01-21T22:24:52.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.343+0000 7f4753e00640 1 -- 172.21.15.138:0/343446525 wait complete. 2024-01-21T22:24:52.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.345+0000 7f4753e00640 1 Processor -- start 2024-01-21T22:24:52.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.345+0000 7f4753e00640 1 -- start start 2024-01-21T22:24:52.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.346+0000 7f4753e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f475406c0a0 0x7f47541f0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:52.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.346+0000 7f4753e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47540cc090 0x7f475415bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:52.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.346+0000 7f4753e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4754153a50 0x7f475414e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:52.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.346+0000 7f4753e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47541f0540 con 0x7f47540cc090 2024-01-21T22:24:52.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.346+0000 7f4753e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47541501b0 con 0x7f475406c0a0 2024-01-21T22:24:52.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.346+0000 7f4753e00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f475414f630 con 0x7f4754153a50 2024-01-21T22:24:52.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.346+0000 7f4752a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f475406c0a0 0x7f47541f0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:52.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.346+0000 7f4753400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4754153a50 0x7f475414e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:52.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.346+0000 7f4752000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47540cc090 0x7f475415bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:52.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.347+0000 7f4752a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f475406c0a0 0x7f47541f0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:43594/0 (socket says 172.21.15.138:43594) 2024-01-21T22:24:52.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.347+0000 7f4752a00640 1 -- 172.21.15.138:0/2700514459 learned_addr learned my addr 172.21.15.138:0/2700514459 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:24:52.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.350+0000 7f4752a00640 1 -- 172.21.15.138:0/2700514459 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4754153a50 msgr2=0x7f475414e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:52.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.350+0000 7f4752a00640 1 --2- 172.21.15.138:0/2700514459 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4754153a50 0x7f475414e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:52.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.350+0000 7f4752a00640 1 -- 172.21.15.138:0/2700514459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47540cc090 msgr2=0x7f475415bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:52.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.350+0000 7f4752a00640 1 --2- 172.21.15.138:0/2700514459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47540cc090 0x7f475415bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:52.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.350+0000 7f4752a00640 1 -- 172.21.15.138:0/2700514459 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c009400 con 0x7f475406c0a0 2024-01-21T22:24:52.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.352+0000 7f4752a00640 1 --2- 172.21.15.138:0/2700514459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f475406c0a0 0x7f47541f0000 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f474400b410 tx=0x7f474400b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:52.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.357+0000 7f474aa00640 1 -- 172.21.15.138:0/2700514459 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4744011830 con 0x7f475406c0a0 2024-01-21T22:24:52.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.357+0000 7f474aa00640 1 -- 172.21.15.138:0/2700514459 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47440119b0 con 0x7f475406c0a0 2024-01-21T22:24:52.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.357+0000 7f474aa00640 1 -- 172.21.15.138:0/2700514459 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4744011d60 con 0x7f475406c0a0 2024-01-21T22:24:52.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.357+0000 7f4753e00640 1 -- 172.21.15.138:0/2700514459 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475414f910 con 0x7f475406c0a0 2024-01-21T22:24:52.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.357+0000 7f4753e00640 1 -- 172.21.15.138:0/2700514459 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475414fed0 con 0x7f475406c0a0 2024-01-21T22:24:52.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.358+0000 7f4753e00640 1 -- 172.21.15.138:0/2700514459 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f475404ea30 con 0x7f475406c0a0 2024-01-21T22:24:52.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.631+0000 7f474aa00640 1 -- 172.21.15.138:0/2700514459 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 89460+0+0 (secure 0 0 0) 0x7f474401a400 con 0x7f475406c0a0 2024-01-21T22:24:52.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.632+0000 7f474aa00640 1 --2- 172.21.15.138:0/2700514459 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f472c06b320 0x7f472c06d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:52.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.632+0000 7f474aa00640 1 -- 172.21.15.138:0/2700514459 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4744089d00 con 0x7f475406c0a0 2024-01-21T22:24:52.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.632+0000 7f4752000640 1 --2- 172.21.15.138:0/2700514459 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f472c06b320 0x7f472c06d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:52.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.633+0000 7f4752000640 1 --2- 172.21.15.138:0/2700514459 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f472c06b320 0x7f472c06d7e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f473c036000 tx=0x7f473c03d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:52.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.641+0000 7f474aa00640 1 -- 172.21.15.138:0/2700514459 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f4744054f50 con 0x7f475406c0a0 2024-01-21T22:24:52.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.874+0000 7f4753e00640 1 -- 172.21.15.138:0/2700514459 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4754151450 con 0x7f475406c0a0 2024-01-21T22:24:52.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.882+0000 7f474aa00640 1 -- 172.21.15.138:0/2700514459 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f4744015090 con 0x7f475406c0a0 2024-01-21T22:24:52.883 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:24:52.883 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-01-21T22:24:52.883 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 1 2024-01-21T22:24:52.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.884+0000 7f4753e00640 1 -- 172.21.15.138:0/2700514459 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f472c06b320 msgr2=0x7f472c06d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:52.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.884+0000 7f4753e00640 1 --2- 172.21.15.138:0/2700514459 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f472c06b320 0x7f472c06d7e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f473c036000 tx=0x7f473c03d040 comp rx=0 tx=0).stop 2024-01-21T22:24:52.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.884+0000 7f4753e00640 1 -- 172.21.15.138:0/2700514459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f475406c0a0 msgr2=0x7f47541f0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:52.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.884+0000 7f4753e00640 1 --2- 172.21.15.138:0/2700514459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f475406c0a0 0x7f47541f0000 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f474400b410 tx=0x7f474400b8d0 comp rx=0 tx=0).stop 2024-01-21T22:24:52.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.884+0000 7f4753e00640 1 -- 172.21.15.138:0/2700514459 shutdown_connections 2024-01-21T22:24:52.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.884+0000 7f4753e00640 1 --2- 172.21.15.138:0/2700514459 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f472c06b320 0x7f472c06d7e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:52.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.884+0000 7f4753e00640 1 --2- 172.21.15.138:0/2700514459 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4754153a50 0x7f475414e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:52.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.884+0000 7f4753e00640 1 --2- 172.21.15.138:0/2700514459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47540cc090 0x7f475415bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:52.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.884+0000 7f4753e00640 1 --2- 172.21.15.138:0/2700514459 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f475406c0a0 0x7f47541f0000 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:52.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.885+0000 7f4753e00640 1 -- 172.21.15.138:0/2700514459 >> 172.21.15.138:0/2700514459 conn(0x7f475406d240 msgr2=0x7f475406b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:52.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.885+0000 7f4753e00640 1 -- 172.21.15.138:0/2700514459 shutdown_connections 2024-01-21T22:24:52.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:52.885+0000 7f4753e00640 1 -- 172.21.15.138:0/2700514459 wait complete. 2024-01-21T22:24:54.792 INFO:tasks.ceph.mds.d.smithi152.stdout:starting mds.d at 2024-01-21T22:24:54.829 INFO:tasks.ceph.mds.b.smithi152.stdout:starting mds.b at 2024-01-21T22:24:54.904 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-21T22:24:54.932 INFO:tasks.ceph.mds.f.smithi152.stdout:starting mds.f at 2024-01-21T22:24:55.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.098+0000 7f3ac2a00640 1 Processor -- start 2024-01-21T22:24:55.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.098+0000 7f3ac2a00640 1 -- start start 2024-01-21T22:24:55.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.100+0000 7f3ac2a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3abc150a00 0x7f3abc0c25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:55.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.100+0000 7f3ac2a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3abc0716e0 0x7f3abc0c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:55.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.100+0000 7f3ac2a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3abc0c7520 0x7f3abc06ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:55.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.100+0000 7f3ac2a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3abc0c4cc0 con 0x7f3abc150a00 2024-01-21T22:24:55.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.100+0000 7f3ac2a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3abc06a920 con 0x7f3abc0716e0 2024-01-21T22:24:55.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.100+0000 7f3ac2a00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc157740 con 0x7f3abc0c7520 2024-01-21T22:24:55.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.101+0000 7f3abb400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3abc0716e0 0x7f3abc0c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:55.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.101+0000 7f3abbe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3abc150a00 0x7f3abc0c25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:55.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.101+0000 7f3ac0a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3abc0c7520 0x7f3abc06ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:55.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.101+0000 7f3abb400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f3abc0716e0 0x7f3abc0c6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:43608/0 (socket says 172.21.15.138:43608) 2024-01-21T22:24:55.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.101+0000 7f3abb400640 1 -- 172.21.15.138:0/715917714 learned_addr learned my addr 172.21.15.138:0/715917714 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:24:55.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.101+0000 7f3abbe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3abc150a00 0x7f3abc0c25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42276/0 (socket says 172.21.15.138:42276) 2024-01-21T22:24:55.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.105+0000 7f3abb400640 1 -- 172.21.15.138:0/715917714 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3abc0c7520 msgr2=0x7f3abc06ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:55.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.105+0000 7f3abb400640 1 --2- 172.21.15.138:0/715917714 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3abc0c7520 0x7f3abc06ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:55.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.105+0000 7f3abb400640 1 -- 172.21.15.138:0/715917714 >> v2:172.21.15.138:3300/0 conn(0x7f3abc150a00 msgr2=0x7f3abc0c25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:55.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.105+0000 7f3abb400640 1 --2- 172.21.15.138:0/715917714 >> v2:172.21.15.138:3300/0 conn(0x7f3abc150a00 0x7f3abc0c25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:55.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.105+0000 7f3abb400640 1 -- 172.21.15.138:0/715917714 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3abc156bc0 con 0x7f3abc0716e0 2024-01-21T22:24:55.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.105+0000 7f3abbe00640 1 --2- 172.21.15.138:0/715917714 >> v2:172.21.15.138:3300/0 conn(0x7f3abc150a00 0x7f3abc0c25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-21T22:24:55.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.108+0000 7f3abb400640 1 --2- 172.21.15.138:0/715917714 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3abc0716e0 0x7f3abc0c6fe0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0720f0 tx=0x7f3aa402b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd96a773253ff5ac server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:55.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.113+0000 7f3abaa00640 1 -- 172.21.15.138:0/715917714 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa40317f0 con 0x7f3abc0716e0 2024-01-21T22:24:55.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.113+0000 7f3abaa00640 1 -- 172.21.15.138:0/715917714 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa4031950 con 0x7f3abc0716e0 2024-01-21T22:24:55.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.113+0000 7f3abaa00640 1 -- 172.21.15.138:0/715917714 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4031ce0 con 0x7f3abc0716e0 2024-01-21T22:24:55.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.114+0000 7f3ac2a00640 1 -- 172.21.15.138:0/715917714 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3abc0716e0 msgr2=0x7f3abc0c6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:55.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.114+0000 7f3ac2a00640 1 --2- 172.21.15.138:0/715917714 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3abc0716e0 0x7f3abc0c6fe0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0720f0 tx=0x7f3aa402b890 comp rx=0 tx=0).stop 2024-01-21T22:24:55.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.114+0000 7f3ac2a00640 1 -- 172.21.15.138:0/715917714 shutdown_connections 2024-01-21T22:24:55.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.114+0000 7f3ac2a00640 1 --2- 172.21.15.138:0/715917714 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3abc0c7520 0x7f3abc06ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:55.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.114+0000 7f3ac2a00640 1 --2- 172.21.15.138:0/715917714 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3abc0716e0 0x7f3abc0c6fe0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:55.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.114+0000 7f3ac2a00640 1 --2- 172.21.15.138:0/715917714 >> v2:172.21.15.138:3300/0 conn(0x7f3abc150a00 0x7f3abc0c25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:55.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.114+0000 7f3ac2a00640 1 -- 172.21.15.138:0/715917714 >> 172.21.15.138:0/715917714 conn(0x7f3abc0cb0b0 msgr2=0x7f3abc0cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:55.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.114+0000 7f3ac2a00640 1 -- 172.21.15.138:0/715917714 shutdown_connections 2024-01-21T22:24:55.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.114+0000 7f3ac2a00640 1 -- 172.21.15.138:0/715917714 wait complete. 2024-01-21T22:24:55.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.116+0000 7f3ac2a00640 1 Processor -- start 2024-01-21T22:24:55.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.116+0000 7f3ac2a00640 1 -- start start 2024-01-21T22:24:55.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.117+0000 7f3ac2a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3abc0716e0 0x7f3abc1f0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:55.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.117+0000 7f3ac2a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3abc0c7520 0x7f3abc152170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:55.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.117+0000 7f3ac2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3abc150a00 0x7f3abc156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:55.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.117+0000 7f3ac2a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc1516a0 con 0x7f3abc150a00 2024-01-21T22:24:55.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.117+0000 7f3ac2a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc1ef860 con 0x7f3abc0c7520 2024-01-21T22:24:55.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.117+0000 7f3ac2a00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc1efde0 con 0x7f3abc0716e0 2024-01-21T22:24:55.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.118+0000 7f3abb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3abc0c7520 0x7f3abc152170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:55.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.118+0000 7f3abbe00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3abc0716e0 0x7f3abc1f0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:55.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.118+0000 7f3ac0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3abc150a00 0x7f3abc156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:55.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.118+0000 7f3abb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3abc0c7520 0x7f3abc152170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:43624/0 (socket says 172.21.15.138:43624) 2024-01-21T22:24:55.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.118+0000 7f3abb400640 1 -- 172.21.15.138:0/2569456301 learned_addr learned my addr 172.21.15.138:0/2569456301 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:24:55.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.122+0000 7f3abb400640 1 -- 172.21.15.138:0/2569456301 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3abc0716e0 msgr2=0x7f3abc1f0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:55.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.122+0000 7f3abb400640 1 --2- 172.21.15.138:0/2569456301 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3abc0716e0 0x7f3abc1f0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:55.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.122+0000 7f3abb400640 1 -- 172.21.15.138:0/2569456301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3abc150a00 msgr2=0x7f3abc156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:55.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.122+0000 7f3abb400640 1 --2- 172.21.15.138:0/2569456301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3abc150a00 0x7f3abc156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:55.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.122+0000 7f3abb400640 1 -- 172.21.15.138:0/2569456301 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac009400 con 0x7f3abc0c7520 2024-01-21T22:24:55.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.124+0000 7f3abb400640 1 --2- 172.21.15.138:0/2569456301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3abc0c7520 0x7f3abc152170 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3aa402b8d0 tx=0x7f3aa40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:55.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.126+0000 7f3ab3e00640 1 -- 172.21.15.138:0/2569456301 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4032890 con 0x7f3abc0c7520 2024-01-21T22:24:55.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.126+0000 7f3ac2a00640 1 -- 172.21.15.138:0/2569456301 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa40094d0 con 0x7f3abc0c7520 2024-01-21T22:24:55.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.126+0000 7f3ac2a00640 1 -- 172.21.15.138:0/2569456301 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3abc0ca9a0 con 0x7f3abc0c7520 2024-01-21T22:24:55.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.128+0000 7f3ac2a00640 1 -- 172.21.15.138:0/2569456301 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3abc069760 con 0x7f3abc0c7520 2024-01-21T22:24:55.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.128+0000 7f3ab3e00640 1 -- 172.21.15.138:0/2569456301 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa4031ef0 con 0x7f3abc0c7520 2024-01-21T22:24:55.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.128+0000 7f3ab3e00640 1 -- 172.21.15.138:0/2569456301 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4032bf0 con 0x7f3abc0c7520 2024-01-21T22:24:55.388 INFO:tasks.ceph.mds.c.smithi138.stdout:starting mds.c at 2024-01-21T22:24:55.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.437+0000 7f3ab3e00640 1 -- 172.21.15.138:0/2569456301 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 4) v1 ==== 89460+0+0 (secure 0 0 0) 0x7f3aa4032d50 con 0x7f3abc0c7520 2024-01-21T22:24:55.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.438+0000 7f3ab3e00640 1 --2- 172.21.15.138:0/2569456301 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f3a9406b2a0 0x7f3a9406d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:55.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.439+0000 7f3abbe00640 1 --2- 172.21.15.138:0/2569456301 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f3a9406b2a0 0x7f3a9406d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:55.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.439+0000 7f3ab3e00640 1 -- 172.21.15.138:0/2569456301 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3aa407ab20 con 0x7f3abc0c7520 2024-01-21T22:24:55.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.439+0000 7f3abbe00640 1 --2- 172.21.15.138:0/2569456301 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f3a9406b2a0 0x7f3a9406d760 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009c30 tx=0x7f3aac009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:55.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.451+0000 7f3ab3e00640 1 -- 172.21.15.138:0/2569456301 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f3aa4076e60 con 0x7f3abc0c7520 2024-01-21T22:24:55.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:55.688+0000 7f3ac2a00640 1 -- 172.21.15.138:0/2569456301 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3abc04eaa0 con 0x7f3abc0c7520 2024-01-21T22:24:55.854 INFO:tasks.ceph.mds.e.smithi138.stdout:starting mds.e at 2024-01-21T22:24:56.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:56.112+0000 7f3ab3e00640 1 -- 172.21.15.138:0/2569456301 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7f3aa407a480 con 0x7f3abc0c7520 2024-01-21T22:24:56.113 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:24:56.113 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-01-21T22:24:56.114 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 1 2024-01-21T22:24:56.114 INFO:tasks.ceph.mds.a.smithi138.stdout:starting mds.a at 2024-01-21T22:24:56.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:56.114+0000 7f3ac2a00640 1 -- 172.21.15.138:0/2569456301 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f3a9406b2a0 msgr2=0x7f3a9406d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:56.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:56.114+0000 7f3ac2a00640 1 --2- 172.21.15.138:0/2569456301 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f3a9406b2a0 0x7f3a9406d760 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009c30 tx=0x7f3aac009190 comp rx=0 tx=0).stop 2024-01-21T22:24:56.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:56.114+0000 7f3ac2a00640 1 -- 172.21.15.138:0/2569456301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3abc0c7520 msgr2=0x7f3abc152170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:56.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:56.114+0000 7f3ac2a00640 1 --2- 172.21.15.138:0/2569456301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3abc0c7520 0x7f3abc152170 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3aa402b8d0 tx=0x7f3aa40046c0 comp rx=0 tx=0).stop 2024-01-21T22:24:56.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:56.114+0000 7f3ac2a00640 1 -- 172.21.15.138:0/2569456301 shutdown_connections 2024-01-21T22:24:56.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:56.114+0000 7f3ac2a00640 1 --2- 172.21.15.138:0/2569456301 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f3a9406b2a0 0x7f3a9406d760 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:56.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:56.114+0000 7f3ac2a00640 1 --2- 172.21.15.138:0/2569456301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3abc150a00 0x7f3abc156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:56.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:56.114+0000 7f3ac2a00640 1 --2- 172.21.15.138:0/2569456301 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3abc0c7520 0x7f3abc152170 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:56.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:56.114+0000 7f3ac2a00640 1 --2- 172.21.15.138:0/2569456301 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3abc0716e0 0x7f3abc1f0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:56.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:56.114+0000 7f3ac2a00640 1 -- 172.21.15.138:0/2569456301 >> 172.21.15.138:0/2569456301 conn(0x7f3abc0cb0b0 msgr2=0x7f3abc0c40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:56.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:56.115+0000 7f3ac2a00640 1 -- 172.21.15.138:0/2569456301 shutdown_connections 2024-01-21T22:24:56.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:56.115+0000 7f3ac2a00640 1 -- 172.21.15.138:0/2569456301 wait complete. 2024-01-21T22:24:58.132 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-21T22:24:58.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.337+0000 7fbb3aa00640 1 Processor -- start 2024-01-21T22:24:58.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.338+0000 7fbb3aa00640 1 -- start start 2024-01-21T22:24:58.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.339+0000 7fbb3aa00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb3406c660 0x7fbb340c4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:58.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.339+0000 7fbb3aa00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb3406af50 con 0x7fbb340cb580 2024-01-21T22:24:58.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.339+0000 7fbb3aa00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb3406b230 con 0x7fbb34153f00 2024-01-21T22:24:58.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.339+0000 7fbb3aa00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3415aaa0 con 0x7fbb3406c660 2024-01-21T22:24:58.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.340+0000 7fbb38a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb3406c660 0x7fbb340c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:58.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.340+0000 7fbb33e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fbb34153f00 0x7fbb3406b6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:53362/0 (socket says 172.21.15.138:53362) 2024-01-21T22:24:58.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.340+0000 7fbb33e00640 1 -- 172.21.15.138:0/1332887200 learned_addr learned my addr 172.21.15.138:0/1332887200 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:24:58.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.343+0000 7fbb38a00640 1 -- 172.21.15.138:0/1332887200 >> v1:172.21.15.152:6789/0 conn(0x7fbb34153f00 legacy=0x7fbb3406b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:58.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.343+0000 7fbb38a00640 1 -- 172.21.15.138:0/1332887200 >> v1:172.21.15.138:6789/0 conn(0x7fbb340cb580 legacy=0x7fbb340cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:58.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.343+0000 7fbb38a00640 1 -- 172.21.15.138:0/1332887200 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb340c4950 con 0x7fbb3406c660 2024-01-21T22:24:58.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.345+0000 7fbb38a00640 1 --2- 172.21.15.138:0/1332887200 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb3406c660 0x7fbb340c4320 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0097d0 tx=0x7fbb2c02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=270e87c6b9ae7e3d server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:58.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.349+0000 7fbb33400640 1 -- 172.21.15.138:0/1332887200 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c031800 con 0x7fbb3406c660 2024-01-21T22:24:58.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.349+0000 7fbb33400640 1 -- 172.21.15.138:0/1332887200 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2c031960 con 0x7fbb3406c660 2024-01-21T22:24:58.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.349+0000 7fbb33400640 1 -- 172.21.15.138:0/1332887200 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c031cf0 con 0x7fbb3406c660 2024-01-21T22:24:58.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.350+0000 7fbb3aa00640 1 -- 172.21.15.138:0/1332887200 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb3406c660 msgr2=0x7fbb340c4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:58.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.350+0000 7fbb3aa00640 1 --2- 172.21.15.138:0/1332887200 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb3406c660 0x7fbb340c4320 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0097d0 tx=0x7fbb2c02b8a0 comp rx=0 tx=0).stop 2024-01-21T22:24:58.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.350+0000 7fbb3aa00640 1 -- 172.21.15.138:0/1332887200 shutdown_connections 2024-01-21T22:24:58.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.350+0000 7fbb3aa00640 1 --2- 172.21.15.138:0/1332887200 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb3406c660 0x7fbb340c4320 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:58.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.350+0000 7fbb3aa00640 1 -- 172.21.15.138:0/1332887200 >> 172.21.15.138:0/1332887200 conn(0x7fbb34066820 msgr2=0x7fbb34066c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:58.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.350+0000 7fbb3aa00640 1 -- 172.21.15.138:0/1332887200 shutdown_connections 2024-01-21T22:24:58.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.350+0000 7fbb3aa00640 1 -- 172.21.15.138:0/1332887200 wait complete. 2024-01-21T22:24:58.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.352+0000 7fbb3aa00640 1 Processor -- start 2024-01-21T22:24:58.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.352+0000 7fbb3aa00640 1 -- start start 2024-01-21T22:24:58.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.353+0000 7fbb3aa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb3406c660 0x7fbb3415b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:58.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.353+0000 7fbb3aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb340cb580 0x7fbb34159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:58.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.353+0000 7fbb3aa00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb34153f00 0x7fbb34149960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:58.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.353+0000 7fbb3aa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3415b9a0 con 0x7fbb340cb580 2024-01-21T22:24:58.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.353+0000 7fbb3aa00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb341543d0 con 0x7fbb3406c660 2024-01-21T22:24:58.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.353+0000 7fbb3aa00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb34154f10 con 0x7fbb34153f00 2024-01-21T22:24:58.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.353+0000 7fbb38a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb34153f00 0x7fbb34149960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:58.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.353+0000 7fbb38a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb34153f00 0x7fbb34149960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:36722/0 (socket says 172.21.15.138:36722) 2024-01-21T22:24:58.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.353+0000 7fbb33e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb3406c660 0x7fbb3415b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:58.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.353+0000 7fbb38a00640 1 -- 172.21.15.138:0/3117907240 learned_addr learned my addr 172.21.15.138:0/3117907240 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:24:58.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.357+0000 7fbb38a00640 1 -- 172.21.15.138:0/3117907240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb3406c660 msgr2=0x7fbb3415b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:58.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.357+0000 7fbb38a00640 1 --2- 172.21.15.138:0/3117907240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb3406c660 0x7fbb3415b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:58.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.357+0000 7fbb38a00640 1 -- 172.21.15.138:0/3117907240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb340cb580 msgr2=0x7fbb34159280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:58.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.357+0000 7fbb38a00640 1 --2- 172.21.15.138:0/3117907240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb340cb580 0x7fbb34159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:58.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.357+0000 7fbb38a00640 1 -- 172.21.15.138:0/3117907240 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb2c009400 con 0x7fbb34153f00 2024-01-21T22:24:58.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.359+0000 7fbb38a00640 1 --2- 172.21.15.138:0/3117907240 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb34153f00 0x7fbb34149960 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0097d0 tx=0x7fbb2c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:58.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.361+0000 7fbb30c00640 1 -- 172.21.15.138:0/3117907240 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c03be70 con 0x7fbb34153f00 2024-01-21T22:24:58.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.361+0000 7fbb3aa00640 1 -- 172.21.15.138:0/3117907240 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb3415bba0 con 0x7fbb34153f00 2024-01-21T22:24:58.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.361+0000 7fbb3aa00640 1 -- 172.21.15.138:0/3117907240 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb341548c0 con 0x7fbb34153f00 2024-01-21T22:24:58.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.362+0000 7fbb3aa00640 1 -- 172.21.15.138:0/3117907240 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb34155050 con 0x7fbb34153f00 2024-01-21T22:24:58.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.571+0000 7fbb30c00640 1 -- 172.21.15.138:0/3117907240 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2c04e450 con 0x7fbb34153f00 2024-01-21T22:24:58.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.611+0000 7fbb30c00640 1 -- 172.21.15.138:0/3117907240 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c04e7c0 con 0x7fbb34153f00 2024-01-21T22:24:58.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.614+0000 7fbb30c00640 1 -- 172.21.15.138:0/3117907240 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fbb2c04e960 con 0x7fbb34153f00 2024-01-21T22:24:58.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.616+0000 7fbb30c00640 1 --2- 172.21.15.138:0/3117907240 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fbb0c06b330 0x7fbb0c06d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:58.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.616+0000 7fbb33e00640 1 --2- 172.21.15.138:0/3117907240 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fbb0c06b330 0x7fbb0c06d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:58.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.616+0000 7fbb30c00640 1 -- 172.21.15.138:0/3117907240 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbb2c02e070 con 0x7fbb34153f00 2024-01-21T22:24:58.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.617+0000 7fbb33e00640 1 --2- 172.21.15.138:0/3117907240 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fbb0c06b330 0x7fbb0c06d7f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbb240012e0 tx=0x7fbb24001990 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:58.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.628+0000 7fbb30c00640 1 -- 172.21.15.138:0/3117907240 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fbb2c076e70 con 0x7fbb34153f00 2024-01-21T22:24:58.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.868+0000 7fbb3aa00640 1 -- 172.21.15.138:0/3117907240 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb34155050 con 0x7fbb34153f00 2024-01-21T22:24:58.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.887+0000 7fbb30c00640 1 -- 172.21.15.138:0/3117907240 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+5289 (secure 0 0 0) 0x7fbb34155050 con 0x7fbb34153f00 2024-01-21T22:24:58.888 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:24:58.888 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/3112363661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":3112363661},{"type":"v1","addr":"172.21.15.152:6837","nonce":3112363661}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4223,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/3222259940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":3222259940},{"type":"v1","addr":"172.21.15.152:6839","nonce":3222259940}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/909784319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":909784319},{"type":"v1","addr":"172.21.15.138:6833","nonce":909784319}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4229,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3318550662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3318550662},{"type":"v1","addr":"172.21.15.138:6835","nonce":3318550662}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2986531362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2986531362},{"type":"v1","addr":"172.21.15.138:6837","nonce":2986531362}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6835/2663179600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2663179600},{"type":"v1","addr":"172.21.15.152:6835","nonce":2663179600}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-01-21T22:24:58.889 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 4 2024-01-21T22:24:58.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.889+0000 7fbb3aa00640 1 -- 172.21.15.138:0/3117907240 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fbb0c06b330 msgr2=0x7fbb0c06d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:58.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.890+0000 7fbb3aa00640 1 --2- 172.21.15.138:0/3117907240 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fbb0c06b330 0x7fbb0c06d7f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbb240012e0 tx=0x7fbb24001990 comp rx=0 tx=0).stop 2024-01-21T22:24:58.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.890+0000 7fbb3aa00640 1 -- 172.21.15.138:0/3117907240 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb34153f00 msgr2=0x7fbb34149960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:58.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.890+0000 7fbb3aa00640 1 --2- 172.21.15.138:0/3117907240 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb34153f00 0x7fbb34149960 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c0097d0 tx=0x7fbb2c0046c0 comp rx=0 tx=0).stop 2024-01-21T22:24:58.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.890+0000 7fbb3aa00640 1 -- 172.21.15.138:0/3117907240 shutdown_connections 2024-01-21T22:24:58.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.890+0000 7fbb3aa00640 1 --2- 172.21.15.138:0/3117907240 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fbb0c06b330 0x7fbb0c06d7f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:58.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.890+0000 7fbb3aa00640 1 --2- 172.21.15.138:0/3117907240 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb34153f00 0x7fbb34149960 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:58.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.890+0000 7fbb3aa00640 1 --2- 172.21.15.138:0/3117907240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb340cb580 0x7fbb34159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:58.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.890+0000 7fbb3aa00640 1 --2- 172.21.15.138:0/3117907240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb3406c660 0x7fbb3415b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:58.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.890+0000 7fbb3aa00640 1 -- 172.21.15.138:0/3117907240 >> 172.21.15.138:0/3117907240 conn(0x7fbb34066820 msgr2=0x7fbb340ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:58.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.890+0000 7fbb3aa00640 1 -- 172.21.15.138:0/3117907240 shutdown_connections 2024-01-21T22:24:58.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:58.890+0000 7fbb3aa00640 1 -- 172.21.15.138:0/3117907240 wait complete. 2024-01-21T22:24:58.909 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-21T22:24:59.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.101+0000 7f4542800640 1 Processor -- start 2024-01-21T22:24:59.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.102+0000 7f4542800640 1 -- start start 2024-01-21T22:24:59.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.103+0000 7f4542800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f453c0cc3d0 0x7f453c159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:59.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.103+0000 7f4542800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f453c15dcd0 con 0x7f453c153be0 2024-01-21T22:24:59.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.103+0000 7f4542800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f453c153fc0 con 0x7f453c15a430 2024-01-21T22:24:59.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.103+0000 7f4542800640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453c067220 con 0x7f453c0cc3d0 2024-01-21T22:24:59.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.104+0000 7f4540a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f453c0cc3d0 0x7f453c159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:59.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.104+0000 7f4541400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f453c153be0 0x7f453c066b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:57950/0 (socket says 172.21.15.138:57950) 2024-01-21T22:24:59.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.104+0000 7f4541400640 1 -- 172.21.15.138:0/2894584538 learned_addr learned my addr 172.21.15.138:0/2894584538 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:24:59.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.106+0000 7f453be00640 1 -- 172.21.15.138:0/2894584538 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 881202792 0 0) 0x7f453c153fc0 con 0x7f453c15a430 2024-01-21T22:24:59.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.106+0000 7f453be00640 1 -- 172.21.15.138:0/2894584538 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f451c003520 con 0x7f453c15a430 2024-01-21T22:24:59.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.106+0000 7f4540a00640 1 -- 172.21.15.138:0/2894584538 >> v1:172.21.15.152:6789/0 conn(0x7f453c15a430 legacy=0x7f453c0cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:59.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.106+0000 7f4540a00640 1 -- 172.21.15.138:0/2894584538 >> v1:172.21.15.138:6789/0 conn(0x7f453c153be0 legacy=0x7f453c066b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:59.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.106+0000 7f4540a00640 1 -- 172.21.15.138:0/2894584538 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453c15d150 con 0x7f453c0cc3d0 2024-01-21T22:24:59.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.108+0000 7f4540a00640 1 --2- 172.21.15.138:0/2894584538 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f453c0cc3d0 0x7f453c159ef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f45240097d0 tx=0x7f452402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe2163a78ba99d60 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:59.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.110+0000 7f453be00640 1 -- 172.21.15.138:0/2894584538 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524031890 con 0x7f453c0cc3d0 2024-01-21T22:24:59.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.112+0000 7f453be00640 1 -- 172.21.15.138:0/2894584538 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45240319f0 con 0x7f453c0cc3d0 2024-01-21T22:24:59.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.112+0000 7f453be00640 1 -- 172.21.15.138:0/2894584538 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4524031d60 con 0x7f453c0cc3d0 2024-01-21T22:24:59.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.113+0000 7f4542800640 1 -- 172.21.15.138:0/2894584538 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f453c0cc3d0 msgr2=0x7f453c159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:59.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.113+0000 7f4542800640 1 --2- 172.21.15.138:0/2894584538 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f453c0cc3d0 0x7f453c159ef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f45240097d0 tx=0x7f452402b930 comp rx=0 tx=0).stop 2024-01-21T22:24:59.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.113+0000 7f4542800640 1 -- 172.21.15.138:0/2894584538 shutdown_connections 2024-01-21T22:24:59.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.113+0000 7f4542800640 1 --2- 172.21.15.138:0/2894584538 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f453c0cc3d0 0x7f453c159ef0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:59.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.113+0000 7f4542800640 1 -- 172.21.15.138:0/2894584538 >> 172.21.15.138:0/2894584538 conn(0x7f453c06cfd0 msgr2=0x7f453c066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:59.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.113+0000 7f4542800640 1 -- 172.21.15.138:0/2894584538 shutdown_connections 2024-01-21T22:24:59.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.113+0000 7f4542800640 1 -- 172.21.15.138:0/2894584538 wait complete. 2024-01-21T22:24:59.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.115+0000 7f4542800640 1 Processor -- start 2024-01-21T22:24:59.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.115+0000 7f4542800640 1 -- start start 2024-01-21T22:24:59.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.116+0000 7f4542800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f453c0cc3d0 0x7f453c1e5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:59.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.116+0000 7f4542800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f453c153be0 0x7f453c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:59.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.116+0000 7f4542800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f453c15a430 0x7f453c1565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:59.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.116+0000 7f4542800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c1f3a30 con 0x7f453c0cc3d0 2024-01-21T22:24:59.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.116+0000 7f4542800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c1e2ba0 con 0x7f453c15a430 2024-01-21T22:24:59.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.116+0000 7f4542800640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453c1e30d0 con 0x7f453c153be0 2024-01-21T22:24:59.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.117+0000 7f4541e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f453c15a430 0x7f453c1565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:59.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.117+0000 7f4540a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f453c153be0 0x7f453c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:59.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.117+0000 7f4541400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f453c0cc3d0 0x7f453c1e5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:59.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.117+0000 7f4540a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f453c153be0 0x7f453c0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:36742/0 (socket says 172.21.15.138:36742) 2024-01-21T22:24:59.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.117+0000 7f4540a00640 1 -- 172.21.15.138:0/358660150 learned_addr learned my addr 172.21.15.138:0/358660150 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:24:59.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.120+0000 7f4541e00640 1 -- 172.21.15.138:0/358660150 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f453c153be0 msgr2=0x7f453c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:59.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.120+0000 7f4541e00640 1 --2- 172.21.15.138:0/358660150 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f453c153be0 0x7f453c0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:59.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.120+0000 7f4541e00640 1 -- 172.21.15.138:0/358660150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f453c0cc3d0 msgr2=0x7f453c1e5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:59.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.120+0000 7f4541e00640 1 --2- 172.21.15.138:0/358660150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f453c0cc3d0 0x7f453c1e5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:59.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.120+0000 7f4541e00640 1 -- 172.21.15.138:0/358660150 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4524009400 con 0x7f453c15a430 2024-01-21T22:24:59.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.123+0000 7f4541e00640 1 --2- 172.21.15.138:0/358660150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f453c15a430 0x7f453c1565d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f453400c3c0 tx=0x7f453400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:59.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.126+0000 7f4539600640 1 -- 172.21.15.138:0/358660150 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45340052d0 con 0x7f453c15a430 2024-01-21T22:24:59.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.126+0000 7f4539600640 1 -- 172.21.15.138:0/358660150 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4534005430 con 0x7f453c15a430 2024-01-21T22:24:59.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.126+0000 7f4539600640 1 -- 172.21.15.138:0/358660150 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534003830 con 0x7f453c15a430 2024-01-21T22:24:59.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.127+0000 7f4542800640 1 -- 172.21.15.138:0/358660150 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f453c1e6320 con 0x7f453c15a430 2024-01-21T22:24:59.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.127+0000 7f4542800640 1 -- 172.21.15.138:0/358660150 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f453c1f3370 con 0x7f453c15a430 2024-01-21T22:24:59.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.128+0000 7f4542800640 1 -- 172.21.15.138:0/358660150 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f453c0c5d40 con 0x7f453c15a430 2024-01-21T22:24:59.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.147+0000 7f4539600640 1 -- 172.21.15.138:0/358660150 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f4534003990 con 0x7f453c15a430 2024-01-21T22:24:59.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.148+0000 7f4539600640 1 --2- 172.21.15.138:0/358660150 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f451406b3b0 0x7f451406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:59.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.148+0000 7f4541400640 1 --2- 172.21.15.138:0/358660150 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f451406b3b0 0x7f451406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:59.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.149+0000 7f4541400640 1 --2- 172.21.15.138:0/358660150 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f451406b3b0 0x7f451406d870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f452c0012e0 tx=0x7f452c001a30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:59.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.150+0000 7f4539600640 1 -- 172.21.15.138:0/358660150 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4534057bd0 con 0x7f453c15a430 2024-01-21T22:24:59.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.409+0000 7f4539600640 1 -- 172.21.15.138:0/358660150 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f4534053f10 con 0x7f453c15a430 2024-01-21T22:24:59.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.614+0000 7f4542800640 1 -- 172.21.15.138:0/358660150 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f453c1f3b70 con 0x7f453c15a430 2024-01-21T22:24:59.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.700+0000 7f4539600640 1 -- 172.21.15.138:0/358660150 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12863 (secure 0 0 0) 0x7f453c1f3b70 con 0x7f453c15a430 2024-01-21T22:24:59.702 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:24:59.702 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":18,"fsid":"d488d446-149f-48d0-89ca-b2830ae72846","created":"2024-01-21T22:24:28.581932+0000","modified":"2024-01-21T22:24:53.700285+0000","last_up_change":"2024-01-21T22:24:47.145799+0000","last_in_change":"2024-01-21T22:24:37.672807+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-21T22:24:49.586178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d812b2d6-c8a3-4015-bfdb-5766b25eae5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6825","nonce":2849203277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6827","nonce":2849203277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6831","nonce":2849203277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6829","nonce":2849203277}]},"public_addr":"172.21.15.138:6825/2849203277","cluster_addr":"172.21.15.138:6827/2849203277","heartbeat_back_addr":"172.21.15.138:6831/2849203277","heartbeat_front_addr":"172.21.15.138:6829/2849203277","state":["exists","up"]},{"osd":1,"uuid":"d8d875df-c840-46cc-b6c0-a847f9c93999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6817","nonce":174521731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6819","nonce":174521731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6823","nonce":174521731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6821","nonce":174521731}]},"public_addr":"172.21.15.138:6817/174521731","cluster_addr":"172.21.15.138:6819/174521731","heartbeat_back_addr":"172.21.15.138:6823/174521731","heartbeat_front_addr":"172.21.15.138:6821/174521731","state":["exists","up"]},{"osd":2,"uuid":"4bd94373-76ca-47a6-870b-16d96d7c9e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6801","nonce":1365838748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6803","nonce":1365838748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6809","nonce":1365838748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6805","nonce":1365838748}]},"public_addr":"172.21.15.138:6801/1365838748","cluster_addr":"172.21.15.138:6803/1365838748","heartbeat_back_addr":"172.21.15.138:6809/1365838748","heartbeat_front_addr":"172.21.15.138:6805/1365838748","state":["exists","up"]},{"osd":3,"uuid":"07267087-d4d2-4e88-ab9b-bb87f6dd061a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6808","nonce":1841873885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6811","nonce":1841873885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6815","nonce":1841873885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6813","nonce":1841873885}]},"public_addr":"172.21.15.138:6808/1841873885","cluster_addr":"172.21.15.138:6811/1841873885","heartbeat_back_addr":"172.21.15.138:6815/1841873885","heartbeat_front_addr":"172.21.15.138:6813/1841873885","state":["exists","up"]},{"osd":4,"uuid":"b47ef758-d783-4168-a9a5-11db3de4ca12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6827","nonce":3960812298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6829","nonce":3960812298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6833","nonce":3960812298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6831","nonce":3960812298}]},"public_addr":"172.21.15.152:6827/3960812298","cluster_addr":"172.21.15.152:6829/3960812298","heartbeat_back_addr":"172.21.15.152:6833/3960812298","heartbeat_front_addr":"172.21.15.152:6831/3960812298","state":["exists","up"]},{"osd":5,"uuid":"0ea5d43e-61d8-4823-815f-9e42c62fae97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6819","nonce":4095234314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6821","nonce":4095234314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6825","nonce":4095234314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6823","nonce":4095234314}]},"public_addr":"172.21.15.152:6819/4095234314","cluster_addr":"172.21.15.152:6821/4095234314","heartbeat_back_addr":"172.21.15.152:6825/4095234314","heartbeat_front_addr":"172.21.15.152:6823/4095234314","state":["exists","up"]},{"osd":6,"uuid":"59ffac36-e2e0-4693-ac82-ad5eccf6224d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6803","nonce":223848127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6805","nonce":223848127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6809","nonce":223848127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6807","nonce":223848127}]},"public_addr":"172.21.15.152:6803/223848127","cluster_addr":"172.21.15.152:6805/223848127","heartbeat_back_addr":"172.21.15.152:6809/223848127","heartbeat_front_addr":"172.21.15.152:6807/223848127","state":["exists","up"]},{"osd":7,"uuid":"4f892c7d-32a5-4960-bf88-4ed7c863cd0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6811","nonce":52509095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6813","nonce":52509095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6817","nonce":52509095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6815","nonce":52509095}]},"public_addr":"172.21.15.152:6811/52509095","cluster_addr":"172.21.15.152:6813/52509095","heartbeat_back_addr":"172.21.15.152:6817/52509095","heartbeat_front_addr":"172.21.15.152:6815/52509095","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.434460+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.374709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.613134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.298148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.242144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.198102+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.093385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.197180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-21T22:24:59.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.705+0000 7f4542800640 1 -- 172.21.15.138:0/358660150 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f451406b3b0 msgr2=0x7f451406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:59.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.705+0000 7f4542800640 1 --2- 172.21.15.138:0/358660150 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f451406b3b0 0x7f451406d870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f452c0012e0 tx=0x7f452c001a30 comp rx=0 tx=0).stop 2024-01-21T22:24:59.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.705+0000 7f4542800640 1 -- 172.21.15.138:0/358660150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f453c15a430 msgr2=0x7f453c1565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:59.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.705+0000 7f4542800640 1 --2- 172.21.15.138:0/358660150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f453c15a430 0x7f453c1565d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f453400c3c0 tx=0x7f453400c880 comp rx=0 tx=0).stop 2024-01-21T22:24:59.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.705+0000 7f4542800640 1 -- 172.21.15.138:0/358660150 shutdown_connections 2024-01-21T22:24:59.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.705+0000 7f4542800640 1 --2- 172.21.15.138:0/358660150 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f451406b3b0 0x7f451406d870 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:59.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.705+0000 7f4542800640 1 --2- 172.21.15.138:0/358660150 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f453c15a430 0x7f453c1565d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:59.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.705+0000 7f4542800640 1 --2- 172.21.15.138:0/358660150 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f453c153be0 0x7f453c0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:59.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.705+0000 7f4542800640 1 --2- 172.21.15.138:0/358660150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f453c0cc3d0 0x7f453c1e5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:59.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.705+0000 7f4542800640 1 -- 172.21.15.138:0/358660150 >> 172.21.15.138:0/358660150 conn(0x7f453c06cfd0 msgr2=0x7f453c151bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:59.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.705+0000 7f4542800640 1 -- 172.21.15.138:0/358660150 shutdown_connections 2024-01-21T22:24:59.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.705+0000 7f4542800640 1 -- 172.21.15.138:0/358660150 wait complete. 2024-01-21T22:24:59.727 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-21T22:24:49.586178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-21T22:24:59.727 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-21T22:24:59.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.934+0000 7f56d6000640 1 Processor -- start 2024-01-21T22:24:59.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.934+0000 7f56d6000640 1 -- start start 2024-01-21T22:24:59.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.935+0000 7f56d6000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f56d00c9bd0 0x7f56d00ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:59.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.935+0000 7f56d6000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d00c7290 con 0x7f56d00c7e30 2024-01-21T22:24:59.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.935+0000 7f56d6000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d015a570 con 0x7f56d0153c60 2024-01-21T22:24:59.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.935+0000 7f56d6000640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d00cb640 con 0x7f56d00c9bd0 2024-01-21T22:24:59.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.936+0000 7f56cea00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f56d00c9bd0 0x7f56d00ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:59.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.936+0000 7f56cf400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f56d0153c60 0x7f56d00c7720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:53390/0 (socket says 172.21.15.138:53390) 2024-01-21T22:24:59.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.936+0000 7f56cf400640 1 -- 172.21.15.138:0/1180515854 learned_addr learned my addr 172.21.15.138:0/1180515854 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:24:59.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.938+0000 7f56cea00640 1 -- 172.21.15.138:0/1180515854 >> v1:172.21.15.152:6789/0 conn(0x7f56d0153c60 legacy=0x7f56d00c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:59.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.938+0000 7f56cea00640 1 -- 172.21.15.138:0/1180515854 >> v1:172.21.15.138:6789/0 conn(0x7f56d00c7e30 legacy=0x7f56d00cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:24:59.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.938+0000 7f56cea00640 1 -- 172.21.15.138:0/1180515854 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d00ca110 con 0x7f56d00c9bd0 2024-01-21T22:24:59.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.940+0000 7f56cea00640 1 --2- 172.21.15.138:0/1180515854 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f56d00c9bd0 0x7f56d00ca8c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f56b80097d0 tx=0x7f56b802b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ab1ce53b3fd0a81 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:59.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.944+0000 7f56ce000640 1 -- 172.21.15.138:0/1180515854 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8031770 con 0x7f56d00c9bd0 2024-01-21T22:24:59.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.944+0000 7f56ce000640 1 -- 172.21.15.138:0/1180515854 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b80318d0 con 0x7f56d00c9bd0 2024-01-21T22:24:59.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.944+0000 7f56ce000640 1 -- 172.21.15.138:0/1180515854 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8031c60 con 0x7f56d00c9bd0 2024-01-21T22:24:59.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.945+0000 7f56d6000640 1 -- 172.21.15.138:0/1180515854 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f56d00c9bd0 msgr2=0x7f56d00ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:59.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.945+0000 7f56d6000640 1 --2- 172.21.15.138:0/1180515854 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f56d00c9bd0 0x7f56d00ca8c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f56b80097d0 tx=0x7f56b802b810 comp rx=0 tx=0).stop 2024-01-21T22:24:59.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.945+0000 7f56d6000640 1 -- 172.21.15.138:0/1180515854 shutdown_connections 2024-01-21T22:24:59.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.945+0000 7f56d6000640 1 --2- 172.21.15.138:0/1180515854 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f56d00c9bd0 0x7f56d00ca8c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:59.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.945+0000 7f56d6000640 1 -- 172.21.15.138:0/1180515854 >> 172.21.15.138:0/1180515854 conn(0x7f56d00cf430 msgr2=0x7f56d00cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:24:59.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.945+0000 7f56d6000640 1 -- 172.21.15.138:0/1180515854 shutdown_connections 2024-01-21T22:24:59.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.945+0000 7f56d6000640 1 -- 172.21.15.138:0/1180515854 wait complete. 2024-01-21T22:24:59.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.946+0000 7f56d6000640 1 Processor -- start 2024-01-21T22:24:59.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.947+0000 7f56d6000640 1 -- start start 2024-01-21T22:24:59.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.948+0000 7f56d6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56d00c7e30 0x7f56d00ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:59.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.948+0000 7f56d6000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f56d00c9bd0 0x7f56d0156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:59.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.948+0000 7f56d6000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56d0153c60 0x7f56d0156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:24:59.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.948+0000 7f56d6000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d00c5220 con 0x7f56d00c7e30 2024-01-21T22:24:59.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.948+0000 7f56d6000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d01f3af0 con 0x7f56d0153c60 2024-01-21T22:24:59.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.948+0000 7f56d6000640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d006b010 con 0x7f56d00c9bd0 2024-01-21T22:24:59.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.948+0000 7f56cea00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f56d00c9bd0 0x7f56d0156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:59.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.948+0000 7f56cfe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56d0153c60 0x7f56d0156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:24:59.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.949+0000 7f56cea00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f56d00c9bd0 0x7f56d0156950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:36770/0 (socket says 172.21.15.138:36770) 2024-01-21T22:24:59.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.949+0000 7f56cea00640 1 -- 172.21.15.138:0/3569035328 learned_addr learned my addr 172.21.15.138:0/3569035328 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:24:59.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.952+0000 7f56cea00640 1 -- 172.21.15.138:0/3569035328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56d0153c60 msgr2=0x7f56d0156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:59.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.952+0000 7f56cea00640 1 --2- 172.21.15.138:0/3569035328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56d0153c60 0x7f56d0156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:59.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.952+0000 7f56cea00640 1 -- 172.21.15.138:0/3569035328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56d00c7e30 msgr2=0x7f56d00ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:24:59.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.952+0000 7f56cea00640 1 --2- 172.21.15.138:0/3569035328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56d00c7e30 0x7f56d00ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:24:59.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.952+0000 7f56cea00640 1 -- 172.21.15.138:0/3569035328 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b8009400 con 0x7f56d00c9bd0 2024-01-21T22:24:59.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.956+0000 7f56cea00640 1 --2- 172.21.15.138:0/3569035328 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f56d00c9bd0 0x7f56d0156950 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f56b80097d0 tx=0x7f56b80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:24:59.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.961+0000 7f56c7400640 1 -- 172.21.15.138:0/3569035328 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8032810 con 0x7f56d00c9bd0 2024-01-21T22:24:59.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.961+0000 7f56c7400640 1 -- 172.21.15.138:0/3569035328 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b8032990 con 0x7f56d00c9bd0 2024-01-21T22:24:59.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.961+0000 7f56c7400640 1 -- 172.21.15.138:0/3569035328 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8032d40 con 0x7f56d00c9bd0 2024-01-21T22:24:59.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.961+0000 7f56d6000640 1 -- 172.21.15.138:0/3569035328 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d01541c0 con 0x7f56d00c9bd0 2024-01-21T22:24:59.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.961+0000 7f56d6000640 1 -- 172.21.15.138:0/3569035328 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d01f2ee0 con 0x7f56d00c9bd0 2024-01-21T22:24:59.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:24:59.962+0000 7f56d6000640 1 -- 172.21.15.138:0/3569035328 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d00c8ea0 con 0x7f56d00c9bd0 2024-01-21T22:25:00.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.207+0000 7f56c7400640 1 -- 172.21.15.138:0/3569035328 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f56b804d3f0 con 0x7f56d00c9bd0 2024-01-21T22:25:00.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.209+0000 7f56c7400640 1 --2- 172.21.15.138:0/3569035328 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f56a806b3b0 0x7f56a806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:00.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.209+0000 7f56c7400640 1 -- 172.21.15.138:0/3569035328 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f56b807a940 con 0x7f56d00c9bd0 2024-01-21T22:25:00.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.209+0000 7f56cf400640 1 --2- 172.21.15.138:0/3569035328 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f56a806b3b0 0x7f56a806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:00.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.210+0000 7f56cf400640 1 --2- 172.21.15.138:0/3569035328 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f56a806b3b0 0x7f56a806d870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f56c0001550 tx=0x7f56c0007a40 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:00.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.219+0000 7f56c7400640 1 -- 172.21.15.138:0/3569035328 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f56b8076580 con 0x7f56d00c9bd0 2024-01-21T22:25:00.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.421+0000 7f56d6000640 1 -- 172.21.15.138:0/3569035328 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f56d0150fb0 con 0x7f56d00c9bd0 2024-01-21T22:25:00.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.442+0000 7f56c7400640 1 -- 172.21.15.138:0/3569035328 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f56b8079ba0 con 0x7f56d00c9bd0 2024-01-21T22:25:00.443 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-01-21T22:25:00.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.445+0000 7f56d6000640 1 -- 172.21.15.138:0/3569035328 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f56a806b3b0 msgr2=0x7f56a806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:00.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.445+0000 7f56d6000640 1 --2- 172.21.15.138:0/3569035328 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f56a806b3b0 0x7f56a806d870 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f56c0001550 tx=0x7f56c0007a40 comp rx=0 tx=0).stop 2024-01-21T22:25:00.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.445+0000 7f56d6000640 1 -- 172.21.15.138:0/3569035328 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f56d00c9bd0 msgr2=0x7f56d0156950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:00.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.445+0000 7f56d6000640 1 --2- 172.21.15.138:0/3569035328 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f56d00c9bd0 0x7f56d0156950 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f56b80097d0 tx=0x7f56b80046c0 comp rx=0 tx=0).stop 2024-01-21T22:25:00.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.445+0000 7f56d6000640 1 -- 172.21.15.138:0/3569035328 shutdown_connections 2024-01-21T22:25:00.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.445+0000 7f56d6000640 1 --2- 172.21.15.138:0/3569035328 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f56a806b3b0 0x7f56a806d870 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:00.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.445+0000 7f56d6000640 1 --2- 172.21.15.138:0/3569035328 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f56d0153c60 0x7f56d0156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:00.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.445+0000 7f56d6000640 1 --2- 172.21.15.138:0/3569035328 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f56d00c9bd0 0x7f56d0156950 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:00.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.445+0000 7f56d6000640 1 --2- 172.21.15.138:0/3569035328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56d00c7e30 0x7f56d00ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:00.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.445+0000 7f56d6000640 1 -- 172.21.15.138:0/3569035328 >> 172.21.15.138:0/3569035328 conn(0x7f56d00cf430 msgr2=0x7f56d014f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:00.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.445+0000 7f56d6000640 1 -- 172.21.15.138:0/3569035328 shutdown_connections 2024-01-21T22:25:00.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.445+0000 7f56d6000640 1 -- 172.21.15.138:0/3569035328 wait complete. 2024-01-21T22:25:00.464 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-01-21T22:25:00.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.663+0000 7fac89800640 1 Processor -- start 2024-01-21T22:25:00.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.664+0000 7fac89800640 1 -- start start 2024-01-21T22:25:00.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.665+0000 7fac89800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fac840c5ab0 0x7fac84071960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:00.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.665+0000 7fac89800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fac8406c310 0x7fac84157790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:00.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.665+0000 7fac89800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac84157cd0 con 0x7fac841519a0 2024-01-21T22:25:00.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.665+0000 7fac89800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac8406d9f0 con 0x7fac840c5ab0 2024-01-21T22:25:00.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.665+0000 7fac89800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac8406d150 con 0x7fac8406c310 2024-01-21T22:25:00.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.666+0000 7fac83400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fac8406c310 0x7fac84157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:00.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.666+0000 7fac82000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fac840c5ab0 0x7fac84071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:00.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.666+0000 7fac82a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fac841519a0 0x7fac840cb2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:53404/0 (socket says 172.21.15.138:53404) 2024-01-21T22:25:00.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.666+0000 7fac82a00640 1 -- 172.21.15.138:0/3172739055 learned_addr learned my addr 172.21.15.138:0/3172739055 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:00.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.670+0000 7fac82000640 1 -- 172.21.15.138:0/3172739055 >> v2:172.21.15.152:3300/0 conn(0x7fac8406c310 msgr2=0x7fac84157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:00.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.670+0000 7fac82000640 1 --2- 172.21.15.138:0/3172739055 >> v2:172.21.15.152:3300/0 conn(0x7fac8406c310 0x7fac84157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:00.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.670+0000 7fac82000640 1 -- 172.21.15.138:0/3172739055 >> v1:172.21.15.152:6789/0 conn(0x7fac841519a0 legacy=0x7fac840cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:00.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.670+0000 7fac82000640 1 -- 172.21.15.138:0/3172739055 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac8406d430 con 0x7fac840c5ab0 2024-01-21T22:25:00.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.672+0000 7fac82000640 1 --2- 172.21.15.138:0/3172739055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac840c5ab0 0x7fac84071960 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fac840c7380 tx=0x7fac6c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1e7b0ecdcbb3aa server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:00.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.677+0000 7fac81600640 1 -- 172.21.15.138:0/3172739055 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c031830 con 0x7fac840c5ab0 2024-01-21T22:25:00.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.677+0000 7fac81600640 1 -- 172.21.15.138:0/3172739055 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6c031990 con 0x7fac840c5ab0 2024-01-21T22:25:00.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.677+0000 7fac81600640 1 -- 172.21.15.138:0/3172739055 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c031d20 con 0x7fac840c5ab0 2024-01-21T22:25:00.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.678+0000 7fac89800640 1 -- 172.21.15.138:0/3172739055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac840c5ab0 msgr2=0x7fac84071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:00.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.678+0000 7fac89800640 1 --2- 172.21.15.138:0/3172739055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac840c5ab0 0x7fac84071960 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fac840c7380 tx=0x7fac6c02b8d0 comp rx=0 tx=0).stop 2024-01-21T22:25:00.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.678+0000 7fac89800640 1 -- 172.21.15.138:0/3172739055 shutdown_connections 2024-01-21T22:25:00.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.678+0000 7fac89800640 1 --2- 172.21.15.138:0/3172739055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac840c5ab0 0x7fac84071960 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:00.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.678+0000 7fac89800640 1 --2- 172.21.15.138:0/3172739055 >> v2:172.21.15.152:3300/0 conn(0x7fac8406c310 0x7fac84157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:00.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.678+0000 7fac89800640 1 -- 172.21.15.138:0/3172739055 >> 172.21.15.138:0/3172739055 conn(0x7fac8406a8d0 msgr2=0x7fac8406ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:00.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.679+0000 7fac89800640 1 -- 172.21.15.138:0/3172739055 shutdown_connections 2024-01-21T22:25:00.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.679+0000 7fac89800640 1 -- 172.21.15.138:0/3172739055 wait complete. 2024-01-21T22:25:00.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.681+0000 7fac89800640 1 Processor -- start 2024-01-21T22:25:00.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.681+0000 7fac89800640 1 -- start start 2024-01-21T22:25:00.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.682+0000 7fac89800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac8406c310 0x7fac840c15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:00.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.683+0000 7fac89800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac840c5ab0 0x7fac841e2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:00.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.683+0000 7fac89800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fac841519a0 0x7fac841527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:00.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.683+0000 7fac89800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac841e1a90 con 0x7fac8406c310 2024-01-21T22:25:00.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.683+0000 7fac89800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac84152460 con 0x7fac840c5ab0 2024-01-21T22:25:00.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.683+0000 7fac89800640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac840c1af0 con 0x7fac841519a0 2024-01-21T22:25:00.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.683+0000 7fac82000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac840c5ab0 0x7fac841e2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:00.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.683+0000 7fac82a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac8406c310 0x7fac840c15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:00.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.683+0000 7fac83400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fac841519a0 0x7fac841527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:00.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.684+0000 7fac82000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac840c5ab0 0x7fac841e2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:44008/0 (socket says 172.21.15.138:44008) 2024-01-21T22:25:00.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.684+0000 7fac82000640 1 -- 172.21.15.138:0/2362035573 learned_addr learned my addr 172.21.15.138:0/2362035573 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:00.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.687+0000 7fac82000640 1 -- 172.21.15.138:0/2362035573 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fac841519a0 msgr2=0x7fac841527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:00.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.687+0000 7fac82000640 1 --2- 172.21.15.138:0/2362035573 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fac841519a0 0x7fac841527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:00.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.687+0000 7fac82000640 1 -- 172.21.15.138:0/2362035573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac8406c310 msgr2=0x7fac840c15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:00.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.687+0000 7fac82000640 1 --2- 172.21.15.138:0/2362035573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac8406c310 0x7fac840c15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:00.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.687+0000 7fac82000640 1 -- 172.21.15.138:0/2362035573 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6c009400 con 0x7fac840c5ab0 2024-01-21T22:25:00.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.690+0000 7fac82000640 1 --2- 172.21.15.138:0/2362035573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac840c5ab0 0x7fac841e2ca0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009860 tx=0x7fac6c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:00.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.693+0000 7fac7aa00640 1 -- 172.21.15.138:0/2362035573 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c03b9a0 con 0x7fac840c5ab0 2024-01-21T22:25:00.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.693+0000 7fac7aa00640 1 -- 172.21.15.138:0/2362035573 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6c03bb00 con 0x7fac840c5ab0 2024-01-21T22:25:00.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.693+0000 7fac7aa00640 1 -- 172.21.15.138:0/2362035573 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6c0433f0 con 0x7fac840c5ab0 2024-01-21T22:25:00.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.694+0000 7fac89800640 1 -- 172.21.15.138:0/2362035573 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac841e0bc0 con 0x7fac840c5ab0 2024-01-21T22:25:00.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.694+0000 7fac89800640 1 -- 172.21.15.138:0/2362035573 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac841e10f0 con 0x7fac840c5ab0 2024-01-21T22:25:00.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.695+0000 7fac89800640 1 -- 172.21.15.138:0/2362035573 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac841e1bd0 con 0x7fac840c5ab0 2024-01-21T22:25:00.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.944+0000 7fac7aa00640 1 -- 172.21.15.138:0/2362035573 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fac6c04c3b0 con 0x7fac840c5ab0 2024-01-21T22:25:00.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.946+0000 7fac7aa00640 1 --2- 172.21.15.138:0/2362035573 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fac5c06b380 0x7fac5c06d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:00.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.947+0000 7fac7aa00640 1 -- 172.21.15.138:0/2362035573 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fac6c0ab750 con 0x7fac840c5ab0 2024-01-21T22:25:00.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.947+0000 7fac82a00640 1 --2- 172.21.15.138:0/2362035573 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fac5c06b380 0x7fac5c06d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:00.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.948+0000 7fac82a00640 1 --2- 172.21.15.138:0/2362035573 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fac5c06b380 0x7fac5c06d840 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fac74005880 tx=0x7fac74009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:00.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:00.957+0000 7fac7aa00640 1 -- 172.21.15.138:0/2362035573 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fac6c076950 con 0x7fac840c5ab0 2024-01-21T22:25:01.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.190+0000 7fac89800640 1 -- 172.21.15.138:0/2362035573 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fac841e1bd0 con 0x7fac840c5ab0 2024-01-21T22:25:01.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.293+0000 7fac7aa00640 1 -- 172.21.15.138:0/2362035573 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7fac841e1bd0 con 0x7fac840c5ab0 2024-01-21T22:25:01.295 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:25:01.295 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-01-21T22:25:01.295 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-01-21T22:25:01.295 INFO:teuthology.orchestra.run.smithi138.stdout: "poolnum": 1, 2024-01-21T22:25:01.295 INFO:teuthology.orchestra.run.smithi138.stdout: "poolname": ".mgr" 2024-01-21T22:25:01.295 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-01-21T22:25:01.295 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-01-21T22:25:01.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.296+0000 7fac89800640 1 -- 172.21.15.138:0/2362035573 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fac5c06b380 msgr2=0x7fac5c06d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:01.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.296+0000 7fac89800640 1 --2- 172.21.15.138:0/2362035573 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fac5c06b380 0x7fac5c06d840 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fac74005880 tx=0x7fac74009040 comp rx=0 tx=0).stop 2024-01-21T22:25:01.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.296+0000 7fac89800640 1 -- 172.21.15.138:0/2362035573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac840c5ab0 msgr2=0x7fac841e2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:01.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.296+0000 7fac89800640 1 --2- 172.21.15.138:0/2362035573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac840c5ab0 0x7fac841e2ca0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fac6c009860 tx=0x7fac6c0046c0 comp rx=0 tx=0).stop 2024-01-21T22:25:01.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.297+0000 7fac89800640 1 -- 172.21.15.138:0/2362035573 shutdown_connections 2024-01-21T22:25:01.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.297+0000 7fac89800640 1 --2- 172.21.15.138:0/2362035573 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fac5c06b380 0x7fac5c06d840 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:01.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.297+0000 7fac89800640 1 --2- 172.21.15.138:0/2362035573 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fac841519a0 0x7fac841527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:01.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.297+0000 7fac89800640 1 --2- 172.21.15.138:0/2362035573 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fac840c5ab0 0x7fac841e2ca0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:01.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.297+0000 7fac89800640 1 --2- 172.21.15.138:0/2362035573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac8406c310 0x7fac840c15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:01.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.297+0000 7fac89800640 1 -- 172.21.15.138:0/2362035573 >> 172.21.15.138:0/2362035573 conn(0x7fac8406a8d0 msgr2=0x7fac84156280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:01.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.297+0000 7fac89800640 1 -- 172.21.15.138:0/2362035573 shutdown_connections 2024-01-21T22:25:01.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.297+0000 7fac89800640 1 -- 172.21.15.138:0/2362035573 wait complete. 2024-01-21T22:25:01.319 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-01-21T22:25:01.319 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-01-21T22:25:01.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.525+0000 7f893f200640 1 Processor -- start 2024-01-21T22:25:01.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.525+0000 7f893f200640 1 -- start start 2024-01-21T22:25:01.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.526+0000 7f893f200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f893806ce80 0x7f89380c5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:01.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.526+0000 7f893f200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f893806d260 0x7f893806bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:01.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.526+0000 7f893f200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8938072aa0 0x7f893806db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:01.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.526+0000 7f893f200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89381543b0 con 0x7f893806d260 2024-01-21T22:25:01.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.526+0000 7f893f200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8938159ce0 con 0x7f893806ce80 2024-01-21T22:25:01.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.527+0000 7f893f200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89380cbb20 con 0x7f8938072aa0 2024-01-21T22:25:01.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.527+0000 7f893d200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8938072aa0 0x7f893806db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:01.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.527+0000 7f8937e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f893806ce80 0x7f89380c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:01.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.527+0000 7f8937400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f893806d260 0x7f893806bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:01.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.528+0000 7f893d200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8938072aa0 0x7f893806db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:36780/0 (socket says 172.21.15.138:36780) 2024-01-21T22:25:01.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.528+0000 7f893d200640 1 -- 172.21.15.138:0/3305022371 learned_addr learned my addr 172.21.15.138:0/3305022371 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:01.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.530+0000 7f893d200640 1 -- 172.21.15.138:0/3305022371 >> v2:172.21.15.152:3300/0 conn(0x7f893806ce80 msgr2=0x7f89380c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:01.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.530+0000 7f893d200640 1 --2- 172.21.15.138:0/3305022371 >> v2:172.21.15.152:3300/0 conn(0x7f893806ce80 0x7f89380c5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:01.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.530+0000 7f893d200640 1 -- 172.21.15.138:0/3305022371 >> v2:172.21.15.138:3300/0 conn(0x7f893806d260 msgr2=0x7f893806bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:01.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.530+0000 7f893d200640 1 --2- 172.21.15.138:0/3305022371 >> v2:172.21.15.138:3300/0 conn(0x7f893806d260 0x7f893806bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:01.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.530+0000 7f893d200640 1 -- 172.21.15.138:0/3305022371 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8938159f80 con 0x7f8938072aa0 2024-01-21T22:25:01.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.532+0000 7f893d200640 1 --2- 172.21.15.138:0/3305022371 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8938072aa0 0x7f893806db70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f893806eb50 tx=0x7f893002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84c28b926daaa392 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:01.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.536+0000 7f8936a00640 1 -- 172.21.15.138:0/3305022371 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893002cb00 con 0x7f8938072aa0 2024-01-21T22:25:01.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.536+0000 7f8936a00640 1 -- 172.21.15.138:0/3305022371 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893002cc60 con 0x7f8938072aa0 2024-01-21T22:25:01.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.536+0000 7f8936a00640 1 -- 172.21.15.138:0/3305022371 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89300343f0 con 0x7f8938072aa0 2024-01-21T22:25:01.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.537+0000 7f893f200640 1 -- 172.21.15.138:0/3305022371 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8938072aa0 msgr2=0x7f893806db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:01.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.537+0000 7f893f200640 1 --2- 172.21.15.138:0/3305022371 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8938072aa0 0x7f893806db70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f893806eb50 tx=0x7f893002bf60 comp rx=0 tx=0).stop 2024-01-21T22:25:01.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.537+0000 7f893f200640 1 -- 172.21.15.138:0/3305022371 shutdown_connections 2024-01-21T22:25:01.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.537+0000 7f893f200640 1 --2- 172.21.15.138:0/3305022371 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8938072aa0 0x7f893806db70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:01.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.537+0000 7f893f200640 1 --2- 172.21.15.138:0/3305022371 >> v2:172.21.15.138:3300/0 conn(0x7f893806d260 0x7f893806bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:01.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.537+0000 7f893f200640 1 --2- 172.21.15.138:0/3305022371 >> v2:172.21.15.152:3300/0 conn(0x7f893806ce80 0x7f89380c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:01.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.537+0000 7f893f200640 1 -- 172.21.15.138:0/3305022371 >> 172.21.15.138:0/3305022371 conn(0x7f893806add0 msgr2=0x7f89380caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:01.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.537+0000 7f893f200640 1 -- 172.21.15.138:0/3305022371 shutdown_connections 2024-01-21T22:25:01.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.537+0000 7f893f200640 1 -- 172.21.15.138:0/3305022371 wait complete. 2024-01-21T22:25:01.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.538+0000 7f893f200640 1 Processor -- start 2024-01-21T22:25:01.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.539+0000 7f893f200640 1 -- start start 2024-01-21T22:25:01.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.539+0000 7f893f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f893806ce80 0x7f89380c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:01.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.539+0000 7f893f200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f893806d260 0x7f89381efdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:01.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.539+0000 7f893f200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8938072aa0 0x7f8938153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:01.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.540+0000 7f893f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8938156f40 con 0x7f893806ce80 2024-01-21T22:25:01.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.540+0000 7f893f200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89380b3980 con 0x7f8938072aa0 2024-01-21T22:25:01.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.540+0000 7f893f200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89381f2970 con 0x7f893806d260 2024-01-21T22:25:01.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.540+0000 7f8937e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f893806ce80 0x7f89380c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:01.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.540+0000 7f8937400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f893806d260 0x7f89381efdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:01.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.540+0000 7f893d200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8938072aa0 0x7f8938153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:01.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.540+0000 7f8937e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f893806ce80 0x7f89380c4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57546/0 (socket says 172.21.15.138:57546) 2024-01-21T22:25:01.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.540+0000 7f8937e00640 1 -- 172.21.15.138:0/2602772489 learned_addr learned my addr 172.21.15.138:0/2602772489 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:01.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.544+0000 7f893d200640 1 -- 172.21.15.138:0/2602772489 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f893806d260 msgr2=0x7f89381efdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:01.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.544+0000 7f893d200640 1 --2- 172.21.15.138:0/2602772489 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f893806d260 0x7f89381efdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:01.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.544+0000 7f893d200640 1 -- 172.21.15.138:0/2602772489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f893806ce80 msgr2=0x7f89380c4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:01.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.544+0000 7f893d200640 1 --2- 172.21.15.138:0/2602772489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f893806ce80 0x7f89380c4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:01.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.544+0000 7f893d200640 1 -- 172.21.15.138:0/2602772489 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8930009b30 con 0x7f8938072aa0 2024-01-21T22:25:01.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.544+0000 7f8937400640 1 --2- 172.21.15.138:0/2602772489 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f893806d260 0x7f89381efdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-21T22:25:01.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.544+0000 7f8937e00640 1 --2- 172.21.15.138:0/2602772489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f893806ce80 0x7f89380c4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-21T22:25:01.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.546+0000 7f893d200640 1 --2- 172.21.15.138:0/2602772489 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8938072aa0 0x7f8938153130 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8930032000 tx=0x7f8930004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:01.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.550+0000 7f892fe00640 1 -- 172.21.15.138:0/2602772489 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8930040670 con 0x7f8938072aa0 2024-01-21T22:25:01.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.550+0000 7f892fe00640 1 -- 172.21.15.138:0/2602772489 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89300407d0 con 0x7f8938072aa0 2024-01-21T22:25:01.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.550+0000 7f892fe00640 1 -- 172.21.15.138:0/2602772489 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8930040bc0 con 0x7f8938072aa0 2024-01-21T22:25:01.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.550+0000 7f893f200640 1 -- 172.21.15.138:0/2602772489 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89381f2c50 con 0x7f8938072aa0 2024-01-21T22:25:01.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.550+0000 7f893f200640 1 -- 172.21.15.138:0/2602772489 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8938157470 con 0x7f8938072aa0 2024-01-21T22:25:01.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.551+0000 7f893f200640 1 -- 172.21.15.138:0/2602772489 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893804eaa0 con 0x7f8938072aa0 2024-01-21T22:25:01.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.798+0000 7f892fe00640 1 -- 172.21.15.138:0/2602772489 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f8930040d20 con 0x7f8938072aa0 2024-01-21T22:25:01.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.800+0000 7f892fe00640 1 --2- 172.21.15.138:0/2602772489 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f891006b380 0x7f891006d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:01.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.800+0000 7f8937e00640 1 --2- 172.21.15.138:0/2602772489 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f891006b380 0x7f891006d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:01.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.801+0000 7f892fe00640 1 -- 172.21.15.138:0/2602772489 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f893007b7a0 con 0x7f8938072aa0 2024-01-21T22:25:01.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.801+0000 7f8937e00640 1 --2- 172.21.15.138:0/2602772489 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f891006b380 0x7f891006d840 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f89280112c0 tx=0x7f8928009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:01.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:01.813+0000 7f892fe00640 1 -- 172.21.15.138:0/2602772489 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f8930077ae0 con 0x7f8938072aa0 2024-01-21T22:25:02.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.015+0000 7f893f200640 1 -- 172.21.15.138:0/2602772489 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f89380742f0 con 0x7f8938072aa0 2024-01-21T22:25:02.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.578+0000 7f892fe00640 1 -- 172.21.15.138:0/2602772489 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8930009510 con 0x7f8938072aa0 2024-01-21T22:25:02.580 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_metadata' created 2024-01-21T22:25:02.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.582+0000 7f893f200640 1 -- 172.21.15.138:0/2602772489 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f891006b380 msgr2=0x7f891006d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:02.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.582+0000 7f893f200640 1 --2- 172.21.15.138:0/2602772489 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f891006b380 0x7f891006d840 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f89280112c0 tx=0x7f8928009190 comp rx=0 tx=0).stop 2024-01-21T22:25:02.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.582+0000 7f893f200640 1 -- 172.21.15.138:0/2602772489 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8938072aa0 msgr2=0x7f8938153130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:02.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.582+0000 7f893f200640 1 --2- 172.21.15.138:0/2602772489 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8938072aa0 0x7f8938153130 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8930032000 tx=0x7f8930004a40 comp rx=0 tx=0).stop 2024-01-21T22:25:02.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.582+0000 7f893f200640 1 -- 172.21.15.138:0/2602772489 shutdown_connections 2024-01-21T22:25:02.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.582+0000 7f893f200640 1 --2- 172.21.15.138:0/2602772489 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f891006b380 0x7f891006d840 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:02.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.582+0000 7f893f200640 1 --2- 172.21.15.138:0/2602772489 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8938072aa0 0x7f8938153130 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:02.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.582+0000 7f893f200640 1 --2- 172.21.15.138:0/2602772489 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f893806d260 0x7f89381efdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:02.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.582+0000 7f893f200640 1 --2- 172.21.15.138:0/2602772489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f893806ce80 0x7f89380c4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:02.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.582+0000 7f893f200640 1 -- 172.21.15.138:0/2602772489 >> 172.21.15.138:0/2602772489 conn(0x7f893806add0 msgr2=0x7f8938158750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:02.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.582+0000 7f893f200640 1 -- 172.21.15.138:0/2602772489 shutdown_connections 2024-01-21T22:25:02.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.582+0000 7f893f200640 1 -- 172.21.15.138:0/2602772489 wait complete. 2024-01-21T22:25:02.608 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-01-21T22:25:02.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.818+0000 7f4216e00640 1 Processor -- start 2024-01-21T22:25:02.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.818+0000 7f4216e00640 1 -- start start 2024-01-21T22:25:02.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.819+0000 7f4216e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4210153c60 0x7f42100c7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:02.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.819+0000 7f4216e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f42100c6900 0x7f42100cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:02.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.819+0000 7f4216e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42100c7290 con 0x7f42100c9bd0 2024-01-21T22:25:02.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.819+0000 7f4216e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f421015a570 con 0x7f42100c6900 2024-01-21T22:25:02.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.819+0000 7f4216e00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42100cb640 con 0x7f4210153c60 2024-01-21T22:25:02.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.820+0000 7f420fe00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4210153c60 0x7f42100c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:02.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.820+0000 7f420fe00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4210153c60 0x7f42100c7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:36796/0 (socket says 172.21.15.138:36796) 2024-01-21T22:25:02.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.821+0000 7f420fe00640 1 -- 172.21.15.138:0/3744747603 learned_addr learned my addr 172.21.15.138:0/3744747603 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:02.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.823+0000 7f420fe00640 1 -- 172.21.15.138:0/3744747603 >> v2:172.21.15.138:3300/0 conn(0x7f42100c6900 msgr2=0x7f42100cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:02.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.823+0000 7f420fe00640 1 --2- 172.21.15.138:0/3744747603 >> v2:172.21.15.138:3300/0 conn(0x7f42100c6900 0x7f42100cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:02.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.823+0000 7f420fe00640 1 -- 172.21.15.138:0/3744747603 >> v1:172.21.15.138:6789/0 conn(0x7f42100c9bd0 legacy=0x7f42100ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:02.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.823+0000 7f420fe00640 1 -- 172.21.15.138:0/3744747603 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42100ca110 con 0x7f4210153c60 2024-01-21T22:25:02.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.825+0000 7f420fe00640 1 --2- 172.21.15.138:0/3744747603 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4210153c60 0x7f42100c7720 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4200009870 tx=0x7f420002b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a1480ece7e8e4b0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:02.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.828+0000 7f420f400640 1 -- 172.21.15.138:0/3744747603 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42000317c0 con 0x7f4210153c60 2024-01-21T22:25:02.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.829+0000 7f420f400640 1 -- 172.21.15.138:0/3744747603 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4200031920 con 0x7f4210153c60 2024-01-21T22:25:02.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.829+0000 7f420f400640 1 -- 172.21.15.138:0/3744747603 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4200031cb0 con 0x7f4210153c60 2024-01-21T22:25:02.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.829+0000 7f4216e00640 1 -- 172.21.15.138:0/3744747603 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4210153c60 msgr2=0x7f42100c7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:02.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.829+0000 7f4216e00640 1 --2- 172.21.15.138:0/3744747603 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4210153c60 0x7f42100c7720 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4200009870 tx=0x7f420002b860 comp rx=0 tx=0).stop 2024-01-21T22:25:02.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.829+0000 7f4216e00640 1 -- 172.21.15.138:0/3744747603 shutdown_connections 2024-01-21T22:25:02.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.829+0000 7f4216e00640 1 --2- 172.21.15.138:0/3744747603 >> v2:172.21.15.138:3300/0 conn(0x7f42100c6900 0x7f42100cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:02.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.829+0000 7f4216e00640 1 --2- 172.21.15.138:0/3744747603 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4210153c60 0x7f42100c7720 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:02.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.829+0000 7f4216e00640 1 -- 172.21.15.138:0/3744747603 >> 172.21.15.138:0/3744747603 conn(0x7f42100cf430 msgr2=0x7f42100cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:02.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.829+0000 7f4216e00640 1 -- 172.21.15.138:0/3744747603 shutdown_connections 2024-01-21T22:25:02.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.830+0000 7f4216e00640 1 -- 172.21.15.138:0/3744747603 wait complete. 2024-01-21T22:25:02.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.831+0000 7f4216e00640 1 Processor -- start 2024-01-21T22:25:02.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.831+0000 7f4216e00640 1 -- start start 2024-01-21T22:25:02.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.832+0000 7f4216e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f42100c6900 0x7f42101efa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:02.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.832+0000 7f4216e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42100c9bd0 0x7f42101f0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:02.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.833+0000 7f4216e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4210153c60 0x7f42100c4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:02.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.833+0000 7f4216e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42100c42f0 con 0x7f4210153c60 2024-01-21T22:25:02.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.833+0000 7f4216e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42101f13e0 con 0x7f42100c9bd0 2024-01-21T22:25:02.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.833+0000 7f4216e00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42100c3390 con 0x7f42100c6900 2024-01-21T22:25:02.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.833+0000 7f4207e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42100c9bd0 0x7f42101f0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:02.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.833+0000 7f420fe00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f42100c6900 0x7f42101efa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:02.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.834+0000 7f420fe00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f42100c6900 0x7f42101efa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:36812/0 (socket says 172.21.15.138:36812) 2024-01-21T22:25:02.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.834+0000 7f420fe00640 1 -- 172.21.15.138:0/3800645649 learned_addr learned my addr 172.21.15.138:0/3800645649 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:02.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.835+0000 7f4214e00640 1 --2- 172.21.15.138:0/3800645649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4210153c60 0x7f42100c4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:02.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.838+0000 7f420fe00640 1 -- 172.21.15.138:0/3800645649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42100c9bd0 msgr2=0x7f42101f0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:02.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.838+0000 7f420fe00640 1 --2- 172.21.15.138:0/3800645649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42100c9bd0 0x7f42101f0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:02.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.838+0000 7f420fe00640 1 -- 172.21.15.138:0/3800645649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4210153c60 msgr2=0x7f42100c4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:02.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.838+0000 7f420fe00640 1 --2- 172.21.15.138:0/3800645649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4210153c60 0x7f42100c4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:02.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.838+0000 7f420fe00640 1 -- 172.21.15.138:0/3800645649 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42000094d0 con 0x7f42100c6900 2024-01-21T22:25:02.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.841+0000 7f420fe00640 1 --2- 172.21.15.138:0/3800645649 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f42100c6900 0x7f42101efa30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f42000090b0 tx=0x7f42000046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:02.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.845+0000 7f420cc00640 1 -- 172.21.15.138:0/3800645649 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4200032960 con 0x7f42100c6900 2024-01-21T22:25:02.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.845+0000 7f420cc00640 1 -- 172.21.15.138:0/3800645649 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4200032ae0 con 0x7f42100c6900 2024-01-21T22:25:02.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.845+0000 7f4216e00640 1 -- 172.21.15.138:0/3800645649 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42101541c0 con 0x7f42100c6900 2024-01-21T22:25:02.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.845+0000 7f420cc00640 1 -- 172.21.15.138:0/3800645649 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4200032e90 con 0x7f42100c6900 2024-01-21T22:25:02.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.845+0000 7f4216e00640 1 -- 172.21.15.138:0/3800645649 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42100c3950 con 0x7f42100c6900 2024-01-21T22:25:02.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.846+0000 7f4216e00640 1 -- 172.21.15.138:0/3800645649 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42100c4430 con 0x7f42100c6900 2024-01-21T22:25:02.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.865+0000 7f420cc00640 1 -- 172.21.15.138:0/3800645649 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f420003b3f0 con 0x7f42100c6900 2024-01-21T22:25:02.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.867+0000 7f420cc00640 1 --2- 172.21.15.138:0/3800645649 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f41e806b380 0x7f41e806d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:02.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.867+0000 7f420cc00640 1 -- 172.21.15.138:0/3800645649 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f42000abfe0 con 0x7f42100c6900 2024-01-21T22:25:02.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.868+0000 7f4207e00640 1 --2- 172.21.15.138:0/3800645649 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f41e806b380 0x7f41e806d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:02.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:02.868+0000 7f4207e00640 1 --2- 172.21.15.138:0/3800645649 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f41e806b380 0x7f41e806d840 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4210067fa0 tx=0x7f41f8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:03.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.105+0000 7f420cc00640 1 -- 172.21.15.138:0/3800645649 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f4200077190 con 0x7f42100c6900 2024-01-21T22:25:03.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.331+0000 7f4216e00640 1 -- 172.21.15.138:0/3800645649 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f42100c88a0 con 0x7f42100c6900 2024-01-21T22:25:03.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.678+0000 7f420cc00640 1 -- 172.21.15.138:0/3800645649 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7f420007a7b0 con 0x7f42100c6900 2024-01-21T22:25:03.680 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_data' created 2024-01-21T22:25:03.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.681+0000 7f4216e00640 1 -- 172.21.15.138:0/3800645649 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f41e806b380 msgr2=0x7f41e806d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:03.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.681+0000 7f4216e00640 1 --2- 172.21.15.138:0/3800645649 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f41e806b380 0x7f41e806d840 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4210067fa0 tx=0x7f41f8009040 comp rx=0 tx=0).stop 2024-01-21T22:25:03.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.681+0000 7f4216e00640 1 -- 172.21.15.138:0/3800645649 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f42100c6900 msgr2=0x7f42101efa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:03.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.681+0000 7f4216e00640 1 --2- 172.21.15.138:0/3800645649 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f42100c6900 0x7f42101efa30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f42000090b0 tx=0x7f42000046c0 comp rx=0 tx=0).stop 2024-01-21T22:25:03.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.682+0000 7f4216e00640 1 -- 172.21.15.138:0/3800645649 shutdown_connections 2024-01-21T22:25:03.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.682+0000 7f4216e00640 1 --2- 172.21.15.138:0/3800645649 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f41e806b380 0x7f41e806d840 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:03.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.682+0000 7f4216e00640 1 --2- 172.21.15.138:0/3800645649 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4210153c60 0x7f42100c4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:03.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.682+0000 7f4216e00640 1 --2- 172.21.15.138:0/3800645649 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f42100c9bd0 0x7f42101f0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:03.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.682+0000 7f4216e00640 1 --2- 172.21.15.138:0/3800645649 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f42100c6900 0x7f42101efa30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:03.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.682+0000 7f4216e00640 1 -- 172.21.15.138:0/3800645649 >> 172.21.15.138:0/3800645649 conn(0x7f42100cf430 msgr2=0x7f4210152bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:03.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.682+0000 7f4216e00640 1 -- 172.21.15.138:0/3800645649 shutdown_connections 2024-01-21T22:25:03.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.682+0000 7f4216e00640 1 -- 172.21.15.138:0/3800645649 wait complete. 2024-01-21T22:25:03.713 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-01-21T22:25:03.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.895+0000 7f75aea00640 1 Processor -- start 2024-01-21T22:25:03.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.895+0000 7f75aea00640 1 -- start start 2024-01-21T22:25:03.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.896+0000 7f75aea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f75a8150b90 0x7f75a80c2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:03.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.896+0000 7f75aea00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a80c75e0 con 0x7f75a8071380 2024-01-21T22:25:03.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.896+0000 7f75aea00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75a80c6d20 con 0x7f75a80719f0 2024-01-21T22:25:03.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.896+0000 7f75aea00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75a8070dc0 con 0x7f75a8150b90 2024-01-21T22:25:03.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.897+0000 7f75aca00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f75a80719f0 0x7f75a8159c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:53406/0 (socket says 172.21.15.138:53406) 2024-01-21T22:25:03.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.897+0000 7f75aca00640 1 -- 172.21.15.138:0/612219306 learned_addr learned my addr 172.21.15.138:0/612219306 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:03.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.925+0000 7f75a7e00640 1 --2- 172.21.15.138:0/612219306 >> v2:172.21.15.138:3300/0 conn(0x7f75a8150b90 0x7f75a80c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:03.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.930+0000 7f75a7e00640 1 -- 172.21.15.138:0/612219306 >> v1:172.21.15.152:6789/0 conn(0x7f75a80719f0 legacy=0x7f75a8159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:03.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.930+0000 7f75a7e00640 1 -- 172.21.15.138:0/612219306 >> v1:172.21.15.138:6789/0 conn(0x7f75a8071380 legacy=0x7f75a80caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:03.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:03.930+0000 7f75a7e00640 1 -- 172.21.15.138:0/612219306 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a80710a0 con 0x7f75a8150b90 2024-01-21T22:25:04.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.041+0000 7f75a7e00640 1 --2- 172.21.15.138:0/612219306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75a8150b90 0x7f75a80c2540 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f75980097a0 tx=0x7f759802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2398fbcd937d00 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:04.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.095+0000 7f75a6a00640 1 -- 172.21.15.138:0/612219306 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7598030610 con 0x7f75a8150b90 2024-01-21T22:25:04.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.095+0000 7f75a6a00640 1 -- 172.21.15.138:0/612219306 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7598030770 con 0x7f75a8150b90 2024-01-21T22:25:04.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.095+0000 7f75a6a00640 1 -- 172.21.15.138:0/612219306 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7598030b00 con 0x7f75a8150b90 2024-01-21T22:25:04.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.097+0000 7f75aea00640 1 -- 172.21.15.138:0/612219306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75a8150b90 msgr2=0x7f75a80c2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:04.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.097+0000 7f75aea00640 1 --2- 172.21.15.138:0/612219306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75a8150b90 0x7f75a80c2540 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f75980097a0 tx=0x7f759802b960 comp rx=0 tx=0).stop 2024-01-21T22:25:04.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.097+0000 7f75aea00640 1 -- 172.21.15.138:0/612219306 shutdown_connections 2024-01-21T22:25:04.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.097+0000 7f75aea00640 1 --2- 172.21.15.138:0/612219306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75a8150b90 0x7f75a80c2540 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:04.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.097+0000 7f75aea00640 1 -- 172.21.15.138:0/612219306 >> 172.21.15.138:0/612219306 conn(0x7f75a8072a20 msgr2=0x7f75a8066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:04.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.097+0000 7f75aea00640 1 -- 172.21.15.138:0/612219306 shutdown_connections 2024-01-21T22:25:04.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.097+0000 7f75aea00640 1 -- 172.21.15.138:0/612219306 wait complete. 2024-01-21T22:25:04.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.101+0000 7f75aea00640 1 Processor -- start 2024-01-21T22:25:04.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.101+0000 7f75aea00640 1 -- start start 2024-01-21T22:25:04.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.103+0000 7f75aea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a8071380 0x7f75a8136f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:04.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.103+0000 7f75aea00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f75a80719f0 0x7f75a81348a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:04.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.103+0000 7f75aea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75a8150b90 0x7f75a812f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:04.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.103+0000 7f75aea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a80cab70 con 0x7f75a8150b90 2024-01-21T22:25:04.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.103+0000 7f75aea00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a80ca6c0 con 0x7f75a8071380 2024-01-21T22:25:04.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.103+0000 7f75aea00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a80cade0 con 0x7f75a80719f0 2024-01-21T22:25:04.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.104+0000 7f75a7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a8071380 0x7f75a8136f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:04.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.104+0000 7f75a7400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f75a80719f0 0x7f75a81348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:04.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.105+0000 7f75a7400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f75a80719f0 0x7f75a81348a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:36824/0 (socket says 172.21.15.138:36824) 2024-01-21T22:25:04.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.105+0000 7f75a7400640 1 -- 172.21.15.138:0/1469263845 learned_addr learned my addr 172.21.15.138:0/1469263845 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:04.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.108+0000 7f75a7400640 1 -- 172.21.15.138:0/1469263845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a8071380 msgr2=0x7f75a8136f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:04.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.108+0000 7f75a7400640 1 --2- 172.21.15.138:0/1469263845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a8071380 0x7f75a8136f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:04.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.108+0000 7f75a7400640 1 -- 172.21.15.138:0/1469263845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75a8150b90 msgr2=0x7f75a812f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:04.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.108+0000 7f75a7400640 1 --2- 172.21.15.138:0/1469263845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75a8150b90 0x7f75a812f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:04.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.108+0000 7f75a7400640 1 -- 172.21.15.138:0/1469263845 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7598009400 con 0x7f75a80719f0 2024-01-21T22:25:04.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.108+0000 7f75a7e00640 1 --2- 172.21.15.138:0/1469263845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a8071380 0x7f75a8136f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-21T22:25:04.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.112+0000 7f75a7400640 1 --2- 172.21.15.138:0/1469263845 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f75a80719f0 0x7f75a81348a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f759000c400 tx=0x7f759000c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:04.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.117+0000 7f759fe00640 1 -- 172.21.15.138:0/1469263845 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75900052d0 con 0x7f75a80719f0 2024-01-21T22:25:04.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.117+0000 7f759fe00640 1 -- 172.21.15.138:0/1469263845 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7590005450 con 0x7f75a80719f0 2024-01-21T22:25:04.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.117+0000 7f75aea00640 1 -- 172.21.15.138:0/1469263845 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a80cb0c0 con 0x7f75a80719f0 2024-01-21T22:25:04.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.117+0000 7f759fe00640 1 -- 172.21.15.138:0/1469263845 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7590007260 con 0x7f75a80719f0 2024-01-21T22:25:04.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.117+0000 7f75aea00640 1 -- 172.21.15.138:0/1469263845 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a8136550 con 0x7f75a80719f0 2024-01-21T22:25:04.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.119+0000 7f75aea00640 1 -- 172.21.15.138:0/1469263845 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a80ca3e0 con 0x7f75a80719f0 2024-01-21T22:25:04.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.139+0000 7f759fe00640 1 -- 172.21.15.138:0/1469263845 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f75900073c0 con 0x7f75a80719f0 2024-01-21T22:25:04.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.140+0000 7f759fe00640 1 --2- 172.21.15.138:0/1469263845 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f758006b380 0x7f758006d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:04.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.140+0000 7f75a7e00640 1 --2- 172.21.15.138:0/1469263845 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f758006b380 0x7f758006d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:04.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.141+0000 7f75a7e00640 1 --2- 172.21.15.138:0/1469263845 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f758006b380 0x7f758006d840 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7598036000 tx=0x7f759803b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:04.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.366+0000 7f759fe00640 1 -- 172.21.15.138:0/1469263845 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f7590007670 con 0x7f75a80719f0 2024-01-21T22:25:04.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.376+0000 7f759fe00640 1 -- 172.21.15.138:0/1469263845 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f7590054380 con 0x7f75a80719f0 2024-01-21T22:25:04.537 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:25:04.545 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:25:04.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:04.624+0000 7f75aea00640 1 -- 172.21.15.138:0/1469263845 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f75a80ca3e0 con 0x7f75a80719f0 2024-01-21T22:25:06.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.210+0000 7f759fe00640 1 -- 172.21.15.138:0/1469263845 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v5) v1 ==== 172+0+0 (secure 0 0 0) 0x7f75a80ca3e0 con 0x7f75a80719f0 2024-01-21T22:25:06.212 INFO:teuthology.orchestra.run.smithi138.stderr:new fs with metadata pool 2 and data pool 3 2024-01-21T22:25:06.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.213+0000 7f75aea00640 1 -- 172.21.15.138:0/1469263845 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f758006b380 msgr2=0x7f758006d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:06.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.213+0000 7f75aea00640 1 --2- 172.21.15.138:0/1469263845 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f758006b380 0x7f758006d840 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7598036000 tx=0x7f759803b040 comp rx=0 tx=0).stop 2024-01-21T22:25:06.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.213+0000 7f75aea00640 1 -- 172.21.15.138:0/1469263845 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f75a80719f0 msgr2=0x7f75a81348a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:06.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.213+0000 7f75aea00640 1 --2- 172.21.15.138:0/1469263845 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f75a80719f0 0x7f75a81348a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f759000c400 tx=0x7f759000c8c0 comp rx=0 tx=0).stop 2024-01-21T22:25:06.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.214+0000 7f75aea00640 1 -- 172.21.15.138:0/1469263845 shutdown_connections 2024-01-21T22:25:06.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.214+0000 7f75aea00640 1 --2- 172.21.15.138:0/1469263845 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f758006b380 0x7f758006d840 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:06.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.214+0000 7f75aea00640 1 --2- 172.21.15.138:0/1469263845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75a8150b90 0x7f75a812f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:06.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.214+0000 7f75aea00640 1 --2- 172.21.15.138:0/1469263845 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f75a80719f0 0x7f75a81348a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:06.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.214+0000 7f75aea00640 1 --2- 172.21.15.138:0/1469263845 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f75a8071380 0x7f75a8136f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:06.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.214+0000 7f75aea00640 1 -- 172.21.15.138:0/1469263845 >> 172.21.15.138:0/1469263845 conn(0x7f75a8072a20 msgr2=0x7f75a80c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:06.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.214+0000 7f75aea00640 1 -- 172.21.15.138:0/1469263845 shutdown_connections 2024-01-21T22:25:06.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.214+0000 7f75aea00640 1 -- 172.21.15.138:0/1469263845 wait complete. 2024-01-21T22:25:06.237 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-21T22:25:06.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.445+0000 7f0803e00640 1 Processor -- start 2024-01-21T22:25:06.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.445+0000 7f0803e00640 1 -- start start 2024-01-21T22:25:06.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.446+0000 7f0803e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0804157160 0x7f0804150890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:06.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.446+0000 7f0803e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0804072c50 con 0x7f08041504b0 2024-01-21T22:25:06.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.446+0000 7f0803e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0804156cd0 con 0x7f08040c02b0 2024-01-21T22:25:06.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.446+0000 7f0803e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0804071ae0 con 0x7f0804157160 2024-01-21T22:25:06.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.447+0000 7f0802a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f08041504b0 0x7f08040c5a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:57988/0 (socket says 172.21.15.138:57988) 2024-01-21T22:25:06.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.447+0000 7f0802a00640 1 -- 172.21.15.138:0/803460198 learned_addr learned my addr 172.21.15.138:0/803460198 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:06.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.447+0000 7f07f9e00640 1 --2- 172.21.15.138:0/803460198 >> v2:172.21.15.152:3300/0 conn(0x7f0804157160 0x7f0804150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:06.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.450+0000 7f0802000640 1 -- 172.21.15.138:0/803460198 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1185473694 0 0) 0x7f0804072c50 con 0x7f08041504b0 2024-01-21T22:25:06.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.450+0000 7f0802000640 1 -- 172.21.15.138:0/803460198 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07e4005730 con 0x7f08041504b0 2024-01-21T22:25:06.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.451+0000 7f07f9e00640 1 -- 172.21.15.138:0/803460198 >> v1:172.21.15.152:6789/0 conn(0x7f08040c02b0 legacy=0x7f08040c1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:06.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.451+0000 7f07f9e00640 1 -- 172.21.15.138:0/803460198 >> v1:172.21.15.138:6789/0 conn(0x7f08041504b0 legacy=0x7f08040c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:06.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.451+0000 7f07f9e00640 1 -- 172.21.15.138:0/803460198 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08040723d0 con 0x7f0804157160 2024-01-21T22:25:06.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.456+0000 7f07f9e00640 1 --2- 172.21.15.138:0/803460198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0804157160 0x7f0804150890 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0097a0 tx=0x7f07ec02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc9d2bfff2f68d41 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:06.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.460+0000 7f0802000640 1 -- 172.21.15.138:0/803460198 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ec030610 con 0x7f0804157160 2024-01-21T22:25:06.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.460+0000 7f0802000640 1 -- 172.21.15.138:0/803460198 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07ec030770 con 0x7f0804157160 2024-01-21T22:25:06.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.460+0000 7f0802000640 1 -- 172.21.15.138:0/803460198 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ec030b00 con 0x7f0804157160 2024-01-21T22:25:06.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.461+0000 7f0803e00640 1 -- 172.21.15.138:0/803460198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0804157160 msgr2=0x7f0804150890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:06.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.461+0000 7f0803e00640 1 --2- 172.21.15.138:0/803460198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0804157160 0x7f0804150890 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0097a0 tx=0x7f07ec02b960 comp rx=0 tx=0).stop 2024-01-21T22:25:06.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.461+0000 7f0803e00640 1 -- 172.21.15.138:0/803460198 shutdown_connections 2024-01-21T22:25:06.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.461+0000 7f0803e00640 1 --2- 172.21.15.138:0/803460198 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0804157160 0x7f0804150890 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:06.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.461+0000 7f0803e00640 1 -- 172.21.15.138:0/803460198 >> 172.21.15.138:0/803460198 conn(0x7f080406dcb0 msgr2=0x7f080406e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:06.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.461+0000 7f0803e00640 1 -- 172.21.15.138:0/803460198 shutdown_connections 2024-01-21T22:25:06.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.461+0000 7f0803e00640 1 -- 172.21.15.138:0/803460198 wait complete. 2024-01-21T22:25:06.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.462+0000 7f0803e00640 1 Processor -- start 2024-01-21T22:25:06.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.463+0000 7f0803e00640 1 -- start start 2024-01-21T22:25:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.463+0000 7f0803e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08040c02b0 0x7f08040ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.463+0000 7f0803e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f08041504b0 0x7f08040cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.464+0000 7f0803e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0804157160 0x7f08040c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.464+0000 7f0803e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080414a940 con 0x7f0804157160 2024-01-21T22:25:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.464+0000 7f0803e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08041ebdc0 con 0x7f08040c02b0 2024-01-21T22:25:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.464+0000 7f0803e00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08040cab80 con 0x7f08041504b0 2024-01-21T22:25:06.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.464+0000 7f0803400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0804157160 0x7f08040c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:06.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.464+0000 7f07f9e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f08041504b0 0x7f08040cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:06.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.464+0000 7f0803400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0804157160 0x7f08040c8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57576/0 (socket says 172.21.15.138:57576) 2024-01-21T22:25:06.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.464+0000 7f0803400640 1 -- 172.21.15.138:0/1103504292 learned_addr learned my addr 172.21.15.138:0/1103504292 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:06.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.466+0000 7f0802a00640 1 --2- 172.21.15.138:0/1103504292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08040c02b0 0x7f08040ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:06.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.468+0000 7f0803400640 1 -- 172.21.15.138:0/1103504292 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f08041504b0 msgr2=0x7f08040cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:06.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.468+0000 7f0803400640 1 --2- 172.21.15.138:0/1103504292 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f08041504b0 0x7f08040cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:06.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.468+0000 7f0803400640 1 -- 172.21.15.138:0/1103504292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08040c02b0 msgr2=0x7f08040ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:06.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.468+0000 7f0803400640 1 --2- 172.21.15.138:0/1103504292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08040c02b0 0x7f08040ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:06.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.468+0000 7f0803400640 1 -- 172.21.15.138:0/1103504292 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ec009400 con 0x7f0804157160 2024-01-21T22:25:06.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.481+0000 7f0803400640 1 --2- 172.21.15.138:0/1103504292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0804157160 0x7f08040c8e70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00c710 tx=0x7f07fc00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:06.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.496+0000 7f07fb400640 1 -- 172.21.15.138:0/1103504292 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc003b70 con 0x7f0804157160 2024-01-21T22:25:06.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.496+0000 7f07fb400640 1 -- 172.21.15.138:0/1103504292 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07fc003cf0 con 0x7f0804157160 2024-01-21T22:25:06.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.496+0000 7f07fb400640 1 -- 172.21.15.138:0/1103504292 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc00aa90 con 0x7f0804157160 2024-01-21T22:25:06.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.496+0000 7f0803e00640 1 -- 172.21.15.138:0/1103504292 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08040ca630 con 0x7f0804157160 2024-01-21T22:25:06.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.496+0000 7f0803e00640 1 -- 172.21.15.138:0/1103504292 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08041ec2b0 con 0x7f0804157160 2024-01-21T22:25:06.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.498+0000 7f0803e00640 1 -- 172.21.15.138:0/1103504292 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08041ebf50 con 0x7f0804157160 2024-01-21T22:25:06.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.958+0000 7f07fb400640 1 -- 172.21.15.138:0/1103504292 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f07fc00abf0 con 0x7f0804157160 2024-01-21T22:25:06.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.960+0000 7f07fb400640 1 --2- 172.21.15.138:0/1103504292 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f07dc06b330 0x7f07dc06d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:06.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.961+0000 7f0802a00640 1 --2- 172.21.15.138:0/1103504292 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f07dc06b330 0x7f07dc06d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:06.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.962+0000 7f0802a00640 1 --2- 172.21.15.138:0/1103504292 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f07dc06b330 0x7f07dc06d7f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f07f40011e0 tx=0x7f07f4007d30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:06.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.988+0000 7f07fb400640 1 -- 172.21.15.138:0/1103504292 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f07fc00ce20 con 0x7f0804157160 2024-01-21T22:25:06.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:06.989+0000 7f07fb400640 1 -- 172.21.15.138:0/1103504292 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f07fc055090 con 0x7f0804157160 2024-01-21T22:25:07.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.215+0000 7f0803e00640 1 -- 172.21.15.138:0/1103504292 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f080404ea30 con 0x7f0804157160 2024-01-21T22:25:07.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.376+0000 7f07fb400640 1 -- 172.21.15.138:0/1103504292 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16522 (secure 0 0 0) 0x7f07fc003e60 con 0x7f0804157160 2024-01-21T22:25:07.379 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:25:07.379 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":23,"fsid":"d488d446-149f-48d0-89ca-b2830ae72846","created":"2024-01-21T22:24:28.581932+0000","modified":"2024-01-21T22:25:06.475790+0000","last_up_change":"2024-01-21T22:24:47.145799+0000","last_in_change":"2024-01-21T22:24:37.672807+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-21T22:24:49.586178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-21T22:25:02.044454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-21T22:25:03.361765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d812b2d6-c8a3-4015-bfdb-5766b25eae5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6825","nonce":2849203277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6827","nonce":2849203277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6831","nonce":2849203277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6829","nonce":2849203277}]},"public_addr":"172.21.15.138:6825/2849203277","cluster_addr":"172.21.15.138:6827/2849203277","heartbeat_back_addr":"172.21.15.138:6831/2849203277","heartbeat_front_addr":"172.21.15.138:6829/2849203277","state":["exists","up"]},{"osd":1,"uuid":"d8d875df-c840-46cc-b6c0-a847f9c93999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6817","nonce":174521731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6819","nonce":174521731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6823","nonce":174521731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6821","nonce":174521731}]},"public_addr":"172.21.15.138:6817/174521731","cluster_addr":"172.21.15.138:6819/174521731","heartbeat_back_addr":"172.21.15.138:6823/174521731","heartbeat_front_addr":"172.21.15.138:6821/174521731","state":["exists","up"]},{"osd":2,"uuid":"4bd94373-76ca-47a6-870b-16d96d7c9e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6801","nonce":1365838748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6803","nonce":1365838748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6809","nonce":1365838748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6805","nonce":1365838748}]},"public_addr":"172.21.15.138:6801/1365838748","cluster_addr":"172.21.15.138:6803/1365838748","heartbeat_back_addr":"172.21.15.138:6809/1365838748","heartbeat_front_addr":"172.21.15.138:6805/1365838748","state":["exists","up"]},{"osd":3,"uuid":"07267087-d4d2-4e88-ab9b-bb87f6dd061a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6808","nonce":1841873885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6811","nonce":1841873885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6815","nonce":1841873885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6813","nonce":1841873885}]},"public_addr":"172.21.15.138:6808/1841873885","cluster_addr":"172.21.15.138:6811/1841873885","heartbeat_back_addr":"172.21.15.138:6815/1841873885","heartbeat_front_addr":"172.21.15.138:6813/1841873885","state":["exists","up"]},{"osd":4,"uuid":"b47ef758-d783-4168-a9a5-11db3de4ca12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6827","nonce":3960812298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6829","nonce":3960812298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6833","nonce":3960812298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6831","nonce":3960812298}]},"public_addr":"172.21.15.152:6827/3960812298","cluster_addr":"172.21.15.152:6829/3960812298","heartbeat_back_addr":"172.21.15.152:6833/3960812298","heartbeat_front_addr":"172.21.15.152:6831/3960812298","state":["exists","up"]},{"osd":5,"uuid":"0ea5d43e-61d8-4823-815f-9e42c62fae97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6819","nonce":4095234314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6821","nonce":4095234314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6825","nonce":4095234314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6823","nonce":4095234314}]},"public_addr":"172.21.15.152:6819/4095234314","cluster_addr":"172.21.15.152:6821/4095234314","heartbeat_back_addr":"172.21.15.152:6825/4095234314","heartbeat_front_addr":"172.21.15.152:6823/4095234314","state":["exists","up"]},{"osd":6,"uuid":"59ffac36-e2e0-4693-ac82-ad5eccf6224d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6803","nonce":223848127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6805","nonce":223848127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6809","nonce":223848127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6807","nonce":223848127}]},"public_addr":"172.21.15.152:6803/223848127","cluster_addr":"172.21.15.152:6805/223848127","heartbeat_back_addr":"172.21.15.152:6809/223848127","heartbeat_front_addr":"172.21.15.152:6807/223848127","state":["exists","up"]},{"osd":7,"uuid":"4f892c7d-32a5-4960-bf88-4ed7c863cd0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6811","nonce":52509095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6813","nonce":52509095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6817","nonce":52509095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6815","nonce":52509095}]},"public_addr":"172.21.15.152:6811/52509095","cluster_addr":"172.21.15.152:6813/52509095","heartbeat_back_addr":"172.21.15.152:6817/52509095","heartbeat_front_addr":"172.21.15.152:6815/52509095","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.434460+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.374709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.613134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.298148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.242144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.198102+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.093385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.197180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-21T22:25:07.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.379+0000 7f0803e00640 1 -- 172.21.15.138:0/1103504292 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f07dc06b330 msgr2=0x7f07dc06d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:07.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.379+0000 7f0803e00640 1 --2- 172.21.15.138:0/1103504292 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f07dc06b330 0x7f07dc06d7f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f07f40011e0 tx=0x7f07f4007d30 comp rx=0 tx=0).stop 2024-01-21T22:25:07.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.379+0000 7f0803e00640 1 -- 172.21.15.138:0/1103504292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0804157160 msgr2=0x7f08040c8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:07.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.379+0000 7f0803e00640 1 --2- 172.21.15.138:0/1103504292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0804157160 0x7f08040c8e70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00c710 tx=0x7f07fc00cbd0 comp rx=0 tx=0).stop 2024-01-21T22:25:07.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.379+0000 7f0803e00640 1 -- 172.21.15.138:0/1103504292 shutdown_connections 2024-01-21T22:25:07.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.379+0000 7f0803e00640 1 --2- 172.21.15.138:0/1103504292 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f07dc06b330 0x7f07dc06d7f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:07.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.379+0000 7f0803e00640 1 --2- 172.21.15.138:0/1103504292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0804157160 0x7f08040c8e70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:07.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.379+0000 7f0803e00640 1 --2- 172.21.15.138:0/1103504292 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f08041504b0 0x7f08040cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:07.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.379+0000 7f0803e00640 1 --2- 172.21.15.138:0/1103504292 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f08040c02b0 0x7f08040ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:07.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.379+0000 7f0803e00640 1 -- 172.21.15.138:0/1103504292 >> 172.21.15.138:0/1103504292 conn(0x7f080406dcb0 msgr2=0x7f080414d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:07.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.379+0000 7f0803e00640 1 -- 172.21.15.138:0/1103504292 shutdown_connections 2024-01-21T22:25:07.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.379+0000 7f0803e00640 1 -- 172.21.15.138:0/1103504292 wait complete. 2024-01-21T22:25:07.401 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-21T22:25:07.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.585+0000 7f7409800640 1 Processor -- start 2024-01-21T22:25:07.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.585+0000 7f7409800640 1 -- start start 2024-01-21T22:25:07.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.586+0000 7f7409800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7404151960 0x7f74040c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:07.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.586+0000 7f7409800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7404070180 0x7f74040c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:07.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.586+0000 7f7409800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7404070e00 con 0x7f7404157570 2024-01-21T22:25:07.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.586+0000 7f7409800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7404070b20 con 0x7f7404151960 2024-01-21T22:25:07.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.586+0000 7f7409800640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74040c61b0 con 0x7f7404070180 2024-01-21T22:25:07.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.587+0000 7f7403400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7404070180 0x7f74040c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:07.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.587+0000 7f7408e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f7404157570 0x7f7404071290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:53432/0 (socket says 172.21.15.138:53432) 2024-01-21T22:25:07.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.587+0000 7f7408e00640 1 -- 172.21.15.138:0/3542149403 learned_addr learned my addr 172.21.15.138:0/3542149403 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:07.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.587+0000 7f7403e00640 1 --2- 172.21.15.138:0/3542149403 >> v2:172.21.15.138:3300/0 conn(0x7f7404151960 0x7f74040c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:07.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.590+0000 7f7402a00640 1 -- 172.21.15.138:0/3542149403 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 571476164 0 0) 0x7f7404070e00 con 0x7f7404157570 2024-01-21T22:25:07.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.590+0000 7f7402a00640 1 -- 172.21.15.138:0/3542149403 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73e4003520 con 0x7f7404157570 2024-01-21T22:25:07.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.590+0000 7f7403400640 1 -- 172.21.15.138:0/3542149403 >> v2:172.21.15.138:3300/0 conn(0x7f7404151960 msgr2=0x7f74040c5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:07.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.590+0000 7f7403400640 1 --2- 172.21.15.138:0/3542149403 >> v2:172.21.15.138:3300/0 conn(0x7f7404151960 0x7f74040c5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:07.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.590+0000 7f7403400640 1 -- 172.21.15.138:0/3542149403 >> v1:172.21.15.152:6789/0 conn(0x7f7404157570 legacy=0x7f7404071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:07.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.590+0000 7f7403400640 1 -- 172.21.15.138:0/3542149403 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7404157ad0 con 0x7f7404070180 2024-01-21T22:25:07.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.592+0000 7f7403400640 1 --2- 172.21.15.138:0/3542149403 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7404070180 0x7f74040c7b00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f73ec0097a0 tx=0x7f73ec02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd7473de5af72957 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:07.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.594+0000 7f7402a00640 1 -- 172.21.15.138:0/3542149403 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec031830 con 0x7f7404070180 2024-01-21T22:25:07.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.595+0000 7f7402a00640 1 -- 172.21.15.138:0/3542149403 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ec031990 con 0x7f7404070180 2024-01-21T22:25:07.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.595+0000 7f7402a00640 1 -- 172.21.15.138:0/3542149403 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ec031d00 con 0x7f7404070180 2024-01-21T22:25:07.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.596+0000 7f7409800640 1 -- 172.21.15.138:0/3542149403 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7404070180 msgr2=0x7f74040c7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:07.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.596+0000 7f7409800640 1 --2- 172.21.15.138:0/3542149403 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7404070180 0x7f74040c7b00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f73ec0097a0 tx=0x7f73ec02b8d0 comp rx=0 tx=0).stop 2024-01-21T22:25:07.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.596+0000 7f7409800640 1 -- 172.21.15.138:0/3542149403 shutdown_connections 2024-01-21T22:25:07.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.596+0000 7f7409800640 1 --2- 172.21.15.138:0/3542149403 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7404070180 0x7f74040c7b00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:07.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.596+0000 7f7409800640 1 --2- 172.21.15.138:0/3542149403 >> v2:172.21.15.138:3300/0 conn(0x7f7404151960 0x7f74040c5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:07.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.596+0000 7f7409800640 1 -- 172.21.15.138:0/3542149403 >> 172.21.15.138:0/3542149403 conn(0x7f740406dcb0 msgr2=0x7f740406e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:07.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.596+0000 7f7409800640 1 -- 172.21.15.138:0/3542149403 shutdown_connections 2024-01-21T22:25:07.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.596+0000 7f7409800640 1 -- 172.21.15.138:0/3542149403 wait complete. 2024-01-21T22:25:07.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.599+0000 7f7409800640 1 Processor -- start 2024-01-21T22:25:07.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.599+0000 7f7409800640 1 -- start start 2024-01-21T22:25:07.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.600+0000 7f7409800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7404070180 0x7f74040c43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:07.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.600+0000 7f7409800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7404151960 0x7f74041f01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:07.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.600+0000 7f7409800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7404157570 0x7f74041e19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:07.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.600+0000 7f7409800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7404150780 con 0x7f7404157570 2024-01-21T22:25:07.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.600+0000 7f7409800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740414e820 con 0x7f7404151960 2024-01-21T22:25:07.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.600+0000 7f7409800640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740414fee0 con 0x7f7404070180 2024-01-21T22:25:07.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.601+0000 7f7403e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7404070180 0x7f74040c43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:07.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.601+0000 7f7403400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7404151960 0x7f74041f01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:07.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.601+0000 7f7408e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7404157570 0x7f74041e19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:07.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.601+0000 7f7403e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7404070180 0x7f74040c43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:36860/0 (socket says 172.21.15.138:36860) 2024-01-21T22:25:07.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.601+0000 7f7403e00640 1 -- 172.21.15.138:0/1299444627 learned_addr learned my addr 172.21.15.138:0/1299444627 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:07.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.604+0000 7f7403e00640 1 -- 172.21.15.138:0/1299444627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7404151960 msgr2=0x7f74041f01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:07.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.604+0000 7f7403e00640 1 --2- 172.21.15.138:0/1299444627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7404151960 0x7f74041f01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:07.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.604+0000 7f7403e00640 1 -- 172.21.15.138:0/1299444627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7404157570 msgr2=0x7f74041e19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:07.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.604+0000 7f7403e00640 1 --2- 172.21.15.138:0/1299444627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7404157570 0x7f74041e19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:07.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.605+0000 7f7403e00640 1 -- 172.21.15.138:0/1299444627 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73ec009400 con 0x7f7404070180 2024-01-21T22:25:07.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.607+0000 7f7403e00640 1 --2- 172.21.15.138:0/1299444627 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7404070180 0x7f74040c43a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f73f400e870 tx=0x7f73f400ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:07.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.611+0000 7f73fbe00640 1 -- 172.21.15.138:0/1299444627 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f40094f0 con 0x7f7404070180 2024-01-21T22:25:07.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.611+0000 7f73fbe00640 1 -- 172.21.15.138:0/1299444627 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f400c940 con 0x7f7404070180 2024-01-21T22:25:07.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.611+0000 7f73fbe00640 1 -- 172.21.15.138:0/1299444627 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f400ccf0 con 0x7f7404070180 2024-01-21T22:25:07.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.611+0000 7f7409800640 1 -- 172.21.15.138:0/1299444627 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74041501c0 con 0x7f7404070180 2024-01-21T22:25:07.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.611+0000 7f7409800640 1 -- 172.21.15.138:0/1299444627 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740414eac0 con 0x7f7404070180 2024-01-21T22:25:07.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.612+0000 7f7409800640 1 -- 172.21.15.138:0/1299444627 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7404151280 con 0x7f7404070180 2024-01-21T22:25:07.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.633+0000 7f73fbe00640 1 -- 172.21.15.138:0/1299444627 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f73f4013470 con 0x7f7404070180 2024-01-21T22:25:07.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.634+0000 7f73fbe00640 1 --2- 172.21.15.138:0/1299444627 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f73dc06b380 0x7f73dc06d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:07.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.634+0000 7f7403400640 1 --2- 172.21.15.138:0/1299444627 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f73dc06b380 0x7f73dc06d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:07.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.635+0000 7f73fbe00640 1 -- 172.21.15.138:0/1299444627 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f73f4058e10 con 0x7f7404070180 2024-01-21T22:25:07.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.635+0000 7f7403400640 1 --2- 172.21.15.138:0/1299444627 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f73dc06b380 0x7f73dc06d840 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f73ec036000 tx=0x7f73ec008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:07.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:07.872+0000 7f73fbe00640 1 -- 172.21.15.138:0/1299444627 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f73f4055150 con 0x7f7404070180 2024-01-21T22:25:08.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.095+0000 7f7409800640 1 -- 172.21.15.138:0/1299444627 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f740404ea30 con 0x7f7404070180 2024-01-21T22:25:08.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.326+0000 7f73fbe00640 1 -- 172.21.15.138:0/1299444627 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16522 (secure 0 0 0) 0x7f73f4017090 con 0x7f7404070180 2024-01-21T22:25:08.329 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:25:08.329 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":23,"fsid":"d488d446-149f-48d0-89ca-b2830ae72846","created":"2024-01-21T22:24:28.581932+0000","modified":"2024-01-21T22:25:06.475790+0000","last_up_change":"2024-01-21T22:24:47.145799+0000","last_in_change":"2024-01-21T22:24:37.672807+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-21T22:24:49.586178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-21T22:25:02.044454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-21T22:25:03.361765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d812b2d6-c8a3-4015-bfdb-5766b25eae5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6825","nonce":2849203277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6827","nonce":2849203277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6831","nonce":2849203277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6829","nonce":2849203277}]},"public_addr":"172.21.15.138:6825/2849203277","cluster_addr":"172.21.15.138:6827/2849203277","heartbeat_back_addr":"172.21.15.138:6831/2849203277","heartbeat_front_addr":"172.21.15.138:6829/2849203277","state":["exists","up"]},{"osd":1,"uuid":"d8d875df-c840-46cc-b6c0-a847f9c93999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6817","nonce":174521731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6819","nonce":174521731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6823","nonce":174521731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6821","nonce":174521731}]},"public_addr":"172.21.15.138:6817/174521731","cluster_addr":"172.21.15.138:6819/174521731","heartbeat_back_addr":"172.21.15.138:6823/174521731","heartbeat_front_addr":"172.21.15.138:6821/174521731","state":["exists","up"]},{"osd":2,"uuid":"4bd94373-76ca-47a6-870b-16d96d7c9e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6801","nonce":1365838748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6803","nonce":1365838748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6809","nonce":1365838748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6805","nonce":1365838748}]},"public_addr":"172.21.15.138:6801/1365838748","cluster_addr":"172.21.15.138:6803/1365838748","heartbeat_back_addr":"172.21.15.138:6809/1365838748","heartbeat_front_addr":"172.21.15.138:6805/1365838748","state":["exists","up"]},{"osd":3,"uuid":"07267087-d4d2-4e88-ab9b-bb87f6dd061a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6808","nonce":1841873885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6811","nonce":1841873885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6815","nonce":1841873885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6813","nonce":1841873885}]},"public_addr":"172.21.15.138:6808/1841873885","cluster_addr":"172.21.15.138:6811/1841873885","heartbeat_back_addr":"172.21.15.138:6815/1841873885","heartbeat_front_addr":"172.21.15.138:6813/1841873885","state":["exists","up"]},{"osd":4,"uuid":"b47ef758-d783-4168-a9a5-11db3de4ca12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6827","nonce":3960812298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6829","nonce":3960812298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6833","nonce":3960812298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6831","nonce":3960812298}]},"public_addr":"172.21.15.152:6827/3960812298","cluster_addr":"172.21.15.152:6829/3960812298","heartbeat_back_addr":"172.21.15.152:6833/3960812298","heartbeat_front_addr":"172.21.15.152:6831/3960812298","state":["exists","up"]},{"osd":5,"uuid":"0ea5d43e-61d8-4823-815f-9e42c62fae97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6819","nonce":4095234314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6821","nonce":4095234314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6825","nonce":4095234314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6823","nonce":4095234314}]},"public_addr":"172.21.15.152:6819/4095234314","cluster_addr":"172.21.15.152:6821/4095234314","heartbeat_back_addr":"172.21.15.152:6825/4095234314","heartbeat_front_addr":"172.21.15.152:6823/4095234314","state":["exists","up"]},{"osd":6,"uuid":"59ffac36-e2e0-4693-ac82-ad5eccf6224d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6803","nonce":223848127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6805","nonce":223848127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6809","nonce":223848127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6807","nonce":223848127}]},"public_addr":"172.21.15.152:6803/223848127","cluster_addr":"172.21.15.152:6805/223848127","heartbeat_back_addr":"172.21.15.152:6809/223848127","heartbeat_front_addr":"172.21.15.152:6807/223848127","state":["exists","up"]},{"osd":7,"uuid":"4f892c7d-32a5-4960-bf88-4ed7c863cd0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6811","nonce":52509095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6813","nonce":52509095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6817","nonce":52509095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6815","nonce":52509095}]},"public_addr":"172.21.15.152:6811/52509095","cluster_addr":"172.21.15.152:6813/52509095","heartbeat_back_addr":"172.21.15.152:6817/52509095","heartbeat_front_addr":"172.21.15.152:6815/52509095","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.434460+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.374709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.613134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.298148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.242144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.198102+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.093385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.197180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-21T22:25:08.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.329+0000 7f7409800640 1 -- 172.21.15.138:0/1299444627 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f73dc06b380 msgr2=0x7f73dc06d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:08.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.329+0000 7f7409800640 1 --2- 172.21.15.138:0/1299444627 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f73dc06b380 0x7f73dc06d840 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f73ec036000 tx=0x7f73ec008030 comp rx=0 tx=0).stop 2024-01-21T22:25:08.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.330+0000 7f7409800640 1 -- 172.21.15.138:0/1299444627 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7404070180 msgr2=0x7f74040c43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:08.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.330+0000 7f7409800640 1 --2- 172.21.15.138:0/1299444627 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7404070180 0x7f74040c43a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f73f400e870 tx=0x7f73f400ed30 comp rx=0 tx=0).stop 2024-01-21T22:25:08.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.330+0000 7f7409800640 1 -- 172.21.15.138:0/1299444627 shutdown_connections 2024-01-21T22:25:08.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.330+0000 7f7409800640 1 --2- 172.21.15.138:0/1299444627 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f73dc06b380 0x7f73dc06d840 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:08.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.330+0000 7f7409800640 1 --2- 172.21.15.138:0/1299444627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7404157570 0x7f74041e19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:08.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.330+0000 7f7409800640 1 --2- 172.21.15.138:0/1299444627 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7404151960 0x7f74041f01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:08.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.330+0000 7f7409800640 1 --2- 172.21.15.138:0/1299444627 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7404070180 0x7f74040c43a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:08.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.330+0000 7f7409800640 1 -- 172.21.15.138:0/1299444627 >> 172.21.15.138:0/1299444627 conn(0x7f740406dcb0 msgr2=0x7f74040c7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:08.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.330+0000 7f7409800640 1 -- 172.21.15.138:0/1299444627 shutdown_connections 2024-01-21T22:25:08.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.330+0000 7f7409800640 1 -- 172.21.15.138:0/1299444627 wait complete. 2024-01-21T22:25:08.353 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-01-21T22:25:08.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.566+0000 7f16c2200640 1 Processor -- start 2024-01-21T22:25:08.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.566+0000 7f16c2200640 1 -- start start 2024-01-21T22:25:08.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.569+0000 7f16c2200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f16bc0c7e30 0x7f16bc0cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:08.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.569+0000 7f16c2200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16bc0c7290 con 0x7f16bc153c60 2024-01-21T22:25:08.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.569+0000 7f16c2200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16bc15a570 con 0x7f16bc0c9bd0 2024-01-21T22:25:08.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.569+0000 7f16c2200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16bc0cb640 con 0x7f16bc0c7e30 2024-01-21T22:25:08.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.569+0000 7f16bbe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f16bc0c7e30 0x7f16bc0cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:08.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.570+0000 7f16bbe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f16bc0c7e30 0x7f16bc0cafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:47720/0 (socket says 172.21.15.138:47720) 2024-01-21T22:25:08.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.570+0000 7f16bbe00640 1 -- 172.21.15.138:0/3746667768 learned_addr learned my addr 172.21.15.138:0/3746667768 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:08.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.572+0000 7f16bbe00640 1 -- 172.21.15.138:0/3746667768 >> v1:172.21.15.152:6789/0 conn(0x7f16bc0c9bd0 legacy=0x7f16bc0ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:08.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.572+0000 7f16bbe00640 1 -- 172.21.15.138:0/3746667768 >> v1:172.21.15.138:6789/0 conn(0x7f16bc153c60 legacy=0x7f16bc0c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:08.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.572+0000 7f16bbe00640 1 -- 172.21.15.138:0/3746667768 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16bc0ca110 con 0x7f16bc0c7e30 2024-01-21T22:25:08.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.576+0000 7f16bbe00640 1 --2- 172.21.15.138:0/3746667768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16bc0c7e30 0x7f16bc0cafd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f16b4009e80 tx=0x7f16b402bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5350c44769caaf3a server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:08.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.578+0000 7f16baa00640 1 -- 172.21.15.138:0/3746667768 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b402ca70 con 0x7f16bc0c7e30 2024-01-21T22:25:08.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.579+0000 7f16baa00640 1 -- 172.21.15.138:0/3746667768 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b402cbd0 con 0x7f16bc0c7e30 2024-01-21T22:25:08.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.579+0000 7f16baa00640 1 -- 172.21.15.138:0/3746667768 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b40343f0 con 0x7f16bc0c7e30 2024-01-21T22:25:08.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.581+0000 7f16c2200640 1 -- 172.21.15.138:0/3746667768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16bc0c7e30 msgr2=0x7f16bc0cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:08.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.581+0000 7f16c2200640 1 --2- 172.21.15.138:0/3746667768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16bc0c7e30 0x7f16bc0cafd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f16b4009e80 tx=0x7f16b402bed0 comp rx=0 tx=0).stop 2024-01-21T22:25:08.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.581+0000 7f16c2200640 1 -- 172.21.15.138:0/3746667768 shutdown_connections 2024-01-21T22:25:08.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.581+0000 7f16c2200640 1 --2- 172.21.15.138:0/3746667768 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16bc0c7e30 0x7f16bc0cafd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:08.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.581+0000 7f16c2200640 1 -- 172.21.15.138:0/3746667768 >> 172.21.15.138:0/3746667768 conn(0x7f16bc0cf430 msgr2=0x7f16bc0cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:08.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.581+0000 7f16c2200640 1 -- 172.21.15.138:0/3746667768 shutdown_connections 2024-01-21T22:25:08.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.581+0000 7f16c2200640 1 -- 172.21.15.138:0/3746667768 wait complete. 2024-01-21T22:25:08.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.584+0000 7f16c2200640 1 Processor -- start 2024-01-21T22:25:08.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.584+0000 7f16c2200640 1 -- start start 2024-01-21T22:25:08.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.586+0000 7f16c2200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16bc0c7e30 0x7f16bc148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:08.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.586+0000 7f16c2200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16bc0c9bd0 0x7f16bc14c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:08.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.586+0000 7f16c2200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16bc153c60 0x7f16bc0c2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:08.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.586+0000 7f16c2200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc1f3df0 con 0x7f16bc153c60 2024-01-21T22:25:08.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.586+0000 7f16c2200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc148ed0 con 0x7f16bc0c7e30 2024-01-21T22:25:08.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.586+0000 7f16c2200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc1f3270 con 0x7f16bc0c9bd0 2024-01-21T22:25:08.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.586+0000 7f16bb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16bc0c7e30 0x7f16bc148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:08.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.587+0000 7f16b3e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16bc0c9bd0 0x7f16bc14c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:08.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.587+0000 7f16bb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16bc0c7e30 0x7f16bc148990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:47728/0 (socket says 172.21.15.138:47728) 2024-01-21T22:25:08.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.587+0000 7f16bb400640 1 -- 172.21.15.138:0/2045571442 learned_addr learned my addr 172.21.15.138:0/2045571442 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:08.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.590+0000 7f16bb400640 1 -- 172.21.15.138:0/2045571442 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16bc0c9bd0 msgr2=0x7f16bc14c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:08.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.590+0000 7f16bb400640 1 --2- 172.21.15.138:0/2045571442 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16bc0c9bd0 0x7f16bc14c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:08.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.590+0000 7f16bb400640 1 -- 172.21.15.138:0/2045571442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16bc153c60 msgr2=0x7f16bc0c2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:08.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.590+0000 7f16bb400640 1 --2- 172.21.15.138:0/2045571442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16bc153c60 0x7f16bc0c2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:08.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.590+0000 7f16bb400640 1 -- 172.21.15.138:0/2045571442 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b4009b30 con 0x7f16bc0c7e30 2024-01-21T22:25:08.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.590+0000 7f16b3e00640 1 --2- 172.21.15.138:0/2045571442 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16bc0c9bd0 0x7f16bc14c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-21T22:25:08.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.593+0000 7f16bb400640 1 --2- 172.21.15.138:0/2045571442 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16bc0c7e30 0x7f16bc148990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00b3e0 tx=0x7f16ac00b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:08.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.597+0000 7f16b3400640 1 -- 172.21.15.138:0/2045571442 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac011830 con 0x7f16bc0c7e30 2024-01-21T22:25:08.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.597+0000 7f16b3400640 1 -- 172.21.15.138:0/2045571442 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ac0119b0 con 0x7f16bc0c7e30 2024-01-21T22:25:08.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.597+0000 7f16b3400640 1 -- 172.21.15.138:0/2045571442 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac011d60 con 0x7f16bc0c7e30 2024-01-21T22:25:08.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.597+0000 7f16c2200640 1 -- 172.21.15.138:0/2045571442 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16bc1541c0 con 0x7f16bc0c7e30 2024-01-21T22:25:08.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.597+0000 7f16c2200640 1 -- 172.21.15.138:0/2045571442 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16bc1f3830 con 0x7f16bc0c7e30 2024-01-21T22:25:08.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.599+0000 7f16c2200640 1 -- 172.21.15.138:0/2045571442 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16bc04eb10 con 0x7f16bc0c7e30 2024-01-21T22:25:08.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.901+0000 7f16b3400640 1 -- 172.21.15.138:0/2045571442 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f16ac01a400 con 0x7f16bc0c7e30 2024-01-21T22:25:08.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.902+0000 7f16b3400640 1 --2- 172.21.15.138:0/2045571442 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f169406b380 0x7f169406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:08.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.902+0000 7f16b3400640 1 -- 172.21.15.138:0/2045571442 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16ac08a200 con 0x7f16bc0c7e30 2024-01-21T22:25:08.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.902+0000 7f16b3e00640 1 --2- 172.21.15.138:0/2045571442 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f169406b380 0x7f169406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:08.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.903+0000 7f16b3e00640 1 --2- 172.21.15.138:0/2045571442 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f169406b380 0x7f169406d840 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f16bc1ed4f0 tx=0x7f16a4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:08.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:08.912+0000 7f16b3400640 1 -- 172.21.15.138:0/2045571442 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f16ac054f70 con 0x7f16bc0c7e30 2024-01-21T22:25:09.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:09.163+0000 7f16c2200640 1 -- 172.21.15.138:0/2045571442 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f16bc0ca6a0 con 0x7f16bc0c7e30 2024-01-21T22:25:09.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:09.915+0000 7f16b3400640 1 -- 172.21.15.138:0/2045571442 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7f16ac058590 con 0x7f16bc0c7e30 2024-01-21T22:25:09.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:09.918+0000 7f16c2200640 1 -- 172.21.15.138:0/2045571442 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f169406b380 msgr2=0x7f169406d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:09.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:09.918+0000 7f16c2200640 1 --2- 172.21.15.138:0/2045571442 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f169406b380 0x7f169406d840 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f16bc1ed4f0 tx=0x7f16a4009040 comp rx=0 tx=0).stop 2024-01-21T22:25:09.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:09.918+0000 7f16c2200640 1 -- 172.21.15.138:0/2045571442 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16bc0c7e30 msgr2=0x7f16bc148990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:09.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:09.918+0000 7f16c2200640 1 --2- 172.21.15.138:0/2045571442 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16bc0c7e30 0x7f16bc148990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f16ac00b3e0 tx=0x7f16ac00b8a0 comp rx=0 tx=0).stop 2024-01-21T22:25:09.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:09.919+0000 7f16c2200640 1 -- 172.21.15.138:0/2045571442 shutdown_connections 2024-01-21T22:25:09.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:09.919+0000 7f16c2200640 1 --2- 172.21.15.138:0/2045571442 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f169406b380 0x7f169406d840 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:09.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:09.919+0000 7f16c2200640 1 --2- 172.21.15.138:0/2045571442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16bc153c60 0x7f16bc0c2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:09.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:09.919+0000 7f16c2200640 1 --2- 172.21.15.138:0/2045571442 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f16bc0c9bd0 0x7f16bc14c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:09.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:09.919+0000 7f16c2200640 1 --2- 172.21.15.138:0/2045571442 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f16bc0c7e30 0x7f16bc148990 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:09.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:09.919+0000 7f16c2200640 1 -- 172.21.15.138:0/2045571442 >> 172.21.15.138:0/2045571442 conn(0x7f16bc0cf430 msgr2=0x7f16bc14b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:09.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:09.919+0000 7f16c2200640 1 -- 172.21.15.138:0/2045571442 shutdown_connections 2024-01-21T22:25:09.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:09.919+0000 7f16c2200640 1 -- 172.21.15.138:0/2045571442 wait complete. 2024-01-21T22:25:09.942 DEBUG:tasks.cephfs.filesystem:fs_config: {'session_timeout': 300} 2024-01-21T22:25:09.942 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-01-21T22:25:10.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.153+0000 7fa8d3600640 1 Processor -- start 2024-01-21T22:25:10.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.154+0000 7fa8d3600640 1 -- start start 2024-01-21T22:25:10.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.155+0000 7fa8d3600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa8cc153f10 0x7fa8cc0c6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:10.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.155+0000 7fa8d3600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fa8cc0cf150 0x7fa8cc159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:10.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.155+0000 7fa8d3600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8cc0cfce0 con 0x7fa8cc0cf570 2024-01-21T22:25:10.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.155+0000 7fa8d3600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8cc15e120 con 0x7fa8cc0cf150 2024-01-21T22:25:10.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.155+0000 7fa8d3600640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc0cfa00 con 0x7fa8cc153f10 2024-01-21T22:25:10.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.156+0000 7fa8d1600640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fa8cc0cf570 0x7fa8cc0ca0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:60618/0 (socket says 172.21.15.138:60618) 2024-01-21T22:25:10.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.156+0000 7fa8d1600640 1 -- 172.21.15.138:0/4023925828 learned_addr learned my addr 172.21.15.138:0/4023925828 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:10.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.156+0000 7fa8cbe00640 1 --2- 172.21.15.138:0/4023925828 >> v2:172.21.15.152:3300/0 conn(0x7fa8cc0cf150 0x7fa8cc159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:10.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.156+0000 7fa8d0c00640 1 --2- 172.21.15.138:0/4023925828 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa8cc153f10 0x7fa8cc0c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:10.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.159+0000 7fa8cb400640 1 -- 172.21.15.138:0/4023925828 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1980463940 0 0) 0x7fa8cc0cfce0 con 0x7fa8cc0cf570 2024-01-21T22:25:10.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.159+0000 7fa8cb400640 1 -- 172.21.15.138:0/4023925828 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8ac003520 con 0x7fa8cc0cf570 2024-01-21T22:25:10.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.159+0000 7fa8cbe00640 1 -- 172.21.15.138:0/4023925828 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa8cc153f10 msgr2=0x7fa8cc0c6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:10.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.159+0000 7fa8cbe00640 1 --2- 172.21.15.138:0/4023925828 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa8cc153f10 0x7fa8cc0c6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:10.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.159+0000 7fa8cbe00640 1 -- 172.21.15.138:0/4023925828 >> v1:172.21.15.138:6789/0 conn(0x7fa8cc0cf570 legacy=0x7fa8cc0ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:10.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.159+0000 7fa8cbe00640 1 -- 172.21.15.138:0/4023925828 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8cc15d360 con 0x7fa8cc0cf150 2024-01-21T22:25:10.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.159+0000 7fa8d0c00640 1 --2- 172.21.15.138:0/4023925828 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa8cc153f10 0x7fa8cc0c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-21T22:25:10.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.161+0000 7fa8cbe00640 1 --2- 172.21.15.138:0/4023925828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8cc0cf150 0x7fa8cc159cc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40097a0 tx=0x7fa8b402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6c8db809e64fd53 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:10.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.165+0000 7fa8cb400640 1 -- 172.21.15.138:0/4023925828 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b402d040 con 0x7fa8cc0cf150 2024-01-21T22:25:10.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.165+0000 7fa8cb400640 1 -- 172.21.15.138:0/4023925828 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b4031810 con 0x7fa8cc0cf150 2024-01-21T22:25:10.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.165+0000 7fa8cb400640 1 -- 172.21.15.138:0/4023925828 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4031ba0 con 0x7fa8cc0cf150 2024-01-21T22:25:10.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.165+0000 7fa8d3600640 1 -- 172.21.15.138:0/4023925828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8cc0cf150 msgr2=0x7fa8cc159cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:10.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.165+0000 7fa8d3600640 1 --2- 172.21.15.138:0/4023925828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8cc0cf150 0x7fa8cc159cc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa8b40097a0 tx=0x7fa8b402b8b0 comp rx=0 tx=0).stop 2024-01-21T22:25:10.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.166+0000 7fa8d3600640 1 -- 172.21.15.138:0/4023925828 shutdown_connections 2024-01-21T22:25:10.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.166+0000 7fa8d3600640 1 --2- 172.21.15.138:0/4023925828 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8cc0cf150 0x7fa8cc159cc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:10.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.166+0000 7fa8d3600640 1 --2- 172.21.15.138:0/4023925828 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa8cc153f10 0x7fa8cc0c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:10.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.166+0000 7fa8d3600640 1 -- 172.21.15.138:0/4023925828 >> 172.21.15.138:0/4023925828 conn(0x7fa8cc06b9c0 msgr2=0x7fa8cc06cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:10.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.166+0000 7fa8d3600640 1 -- 172.21.15.138:0/4023925828 shutdown_connections 2024-01-21T22:25:10.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.166+0000 7fa8d3600640 1 -- 172.21.15.138:0/4023925828 wait complete. 2024-01-21T22:25:10.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.168+0000 7fa8d3600640 1 Processor -- start 2024-01-21T22:25:10.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.168+0000 7fa8d3600640 1 -- start start 2024-01-21T22:25:10.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.169+0000 7fa8d3600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8cc0cf150 0x7fa8cc1e5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:10.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.169+0000 7fa8d3600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa8cc153f10 0x7fa8cc1f3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:10.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.169+0000 7fa8d3600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8cc1f0700 0x7fa8cc152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:10.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.169+0000 7fa8d3600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc1f2620 con 0x7fa8cc0cf150 2024-01-21T22:25:10.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.169+0000 7fa8d3600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc1e5e00 con 0x7fa8cc1f0700 2024-01-21T22:25:10.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.169+0000 7fa8d3600640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc1e42b0 con 0x7fa8cc153f10 2024-01-21T22:25:10.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.169+0000 7fa8d0c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8cc0cf150 0x7fa8cc1e5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:10.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.169+0000 7fa8d1600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8cc1f0700 0x7fa8cc152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:10.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.170+0000 7fa8d0c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8cc0cf150 0x7fa8cc1e5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43480/0 (socket says 172.21.15.138:43480) 2024-01-21T22:25:10.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.170+0000 7fa8d0c00640 1 -- 172.21.15.138:0/3099242634 learned_addr learned my addr 172.21.15.138:0/3099242634 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:10.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.170+0000 7fa8cbe00640 1 --2- 172.21.15.138:0/3099242634 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa8cc153f10 0x7fa8cc1f3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:10.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.173+0000 7fa8d1600640 1 -- 172.21.15.138:0/3099242634 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa8cc153f10 msgr2=0x7fa8cc1f3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:10.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.173+0000 7fa8d1600640 1 --2- 172.21.15.138:0/3099242634 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa8cc153f10 0x7fa8cc1f3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:10.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.173+0000 7fa8d1600640 1 -- 172.21.15.138:0/3099242634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8cc0cf150 msgr2=0x7fa8cc1e5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:10.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.173+0000 7fa8d1600640 1 --2- 172.21.15.138:0/3099242634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8cc0cf150 0x7fa8cc1e5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:10.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.173+0000 7fa8d1600640 1 -- 172.21.15.138:0/3099242634 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8bc009400 con 0x7fa8cc1f0700 2024-01-21T22:25:10.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.173+0000 7fa8cbe00640 1 --2- 172.21.15.138:0/3099242634 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa8cc153f10 0x7fa8cc1f3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-21T22:25:10.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.176+0000 7fa8d1600640 1 --2- 172.21.15.138:0/3099242634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8cc1f0700 0x7fa8cc152b20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400ca50 tx=0x7fa8c400cf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:10.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.179+0000 7fa8c8c00640 1 -- 172.21.15.138:0/3099242634 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c400ab50 con 0x7fa8cc1f0700 2024-01-21T22:25:10.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.179+0000 7fa8c8c00640 1 -- 172.21.15.138:0/3099242634 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c400acb0 con 0x7fa8cc1f0700 2024-01-21T22:25:10.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.179+0000 7fa8c8c00640 1 -- 172.21.15.138:0/3099242634 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4012600 con 0x7fa8cc1f0700 2024-01-21T22:25:10.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.180+0000 7fa8d3600640 1 -- 172.21.15.138:0/3099242634 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8b4009400 con 0x7fa8cc1f0700 2024-01-21T22:25:10.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.180+0000 7fa8d3600640 1 -- 172.21.15.138:0/3099242634 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8cc1f3750 con 0x7fa8cc1f0700 2024-01-21T22:25:10.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.181+0000 7fa8d3600640 1 -- 172.21.15.138:0/3099242634 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8cc0cb370 con 0x7fa8cc1f0700 2024-01-21T22:25:10.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.200+0000 7fa8c8c00640 1 -- 172.21.15.138:0/3099242634 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fa8c4007070 con 0x7fa8cc1f0700 2024-01-21T22:25:10.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.202+0000 7fa8c8c00640 1 --2- 172.21.15.138:0/3099242634 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fa8a406b380 0x7fa8a406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:10.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.203+0000 7fa8d0c00640 1 --2- 172.21.15.138:0/3099242634 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fa8a406b380 0x7fa8a406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:10.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.203+0000 7fa8c8c00640 1 -- 172.21.15.138:0/3099242634 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8c40587a0 con 0x7fa8cc1f0700 2024-01-21T22:25:10.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.203+0000 7fa8d0c00640 1 --2- 172.21.15.138:0/3099242634 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fa8a406b380 0x7fa8a406d840 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc009dd0 tx=0x7fa8bc009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:10.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.437+0000 7fa8c8c00640 1 -- 172.21.15.138:0/3099242634 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fa8c4054ae0 con 0x7fa8cc1f0700 2024-01-21T22:25:10.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:10.688+0000 7fa8d3600640 1 -- 172.21.15.138:0/3099242634 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa8cc1f3750 con 0x7fa8cc1f0700 2024-01-21T22:25:11.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:11.786+0000 7fa8c8c00640 1 -- 172.21.15.138:0/3099242634 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa8cc1f3750 con 0x7fa8cc1f0700 2024-01-21T22:25:11.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:11.789+0000 7fa8d3600640 1 -- 172.21.15.138:0/3099242634 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fa8a406b380 msgr2=0x7fa8a406d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:11.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:11.789+0000 7fa8d3600640 1 --2- 172.21.15.138:0/3099242634 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fa8a406b380 0x7fa8a406d840 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc009dd0 tx=0x7fa8bc009190 comp rx=0 tx=0).stop 2024-01-21T22:25:11.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:11.789+0000 7fa8d3600640 1 -- 172.21.15.138:0/3099242634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8cc1f0700 msgr2=0x7fa8cc152b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:11.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:11.789+0000 7fa8d3600640 1 --2- 172.21.15.138:0/3099242634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8cc1f0700 0x7fa8cc152b20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400ca50 tx=0x7fa8c400cf10 comp rx=0 tx=0).stop 2024-01-21T22:25:11.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:11.790+0000 7fa8d3600640 1 -- 172.21.15.138:0/3099242634 shutdown_connections 2024-01-21T22:25:11.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:11.790+0000 7fa8d3600640 1 --2- 172.21.15.138:0/3099242634 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fa8a406b380 0x7fa8a406d840 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:11.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:11.790+0000 7fa8d3600640 1 --2- 172.21.15.138:0/3099242634 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fa8cc1f0700 0x7fa8cc152b20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:11.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:11.790+0000 7fa8d3600640 1 --2- 172.21.15.138:0/3099242634 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fa8cc153f10 0x7fa8cc1f3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:11.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:11.790+0000 7fa8d3600640 1 --2- 172.21.15.138:0/3099242634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8cc0cf150 0x7fa8cc1e5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:11.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:11.790+0000 7fa8d3600640 1 -- 172.21.15.138:0/3099242634 >> 172.21.15.138:0/3099242634 conn(0x7fa8cc06b9c0 msgr2=0x7fa8cc14c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:11.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:11.790+0000 7fa8d3600640 1 -- 172.21.15.138:0/3099242634 shutdown_connections 2024-01-21T22:25:11.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:11.790+0000 7fa8d3600640 1 -- 172.21.15.138:0/3099242634 wait complete. 2024-01-21T22:25:11.815 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-01-21T22:25:12.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.022+0000 7fe2e5400640 1 Processor -- start 2024-01-21T22:25:12.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.022+0000 7fe2e5400640 1 -- start start 2024-01-21T22:25:12.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.024+0000 7fe2e5400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2e0153e40 0x7fe2e006a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:12.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.024+0000 7fe2e5400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe2e00c6160 0x7fe2e00c6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:12.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.024+0000 7fe2e5400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e00711c0 con 0x7fe2e0071ef0 2024-01-21T22:25:12.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.024+0000 7fe2e5400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e00723a0 con 0x7fe2e00c6160 2024-01-21T22:25:12.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.024+0000 7fe2e5400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e0154220 con 0x7fe2e0153e40 2024-01-21T22:25:12.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.025+0000 7fe2df200640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fe2e0071ef0 0x7fe2e00c70a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:60630/0 (socket says 172.21.15.138:60630) 2024-01-21T22:25:12.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.025+0000 7fe2df200640 1 -- 172.21.15.138:0/3991943872 learned_addr learned my addr 172.21.15.138:0/3991943872 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:12.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.027+0000 7fe2dde00640 1 -- 172.21.15.138:0/3991943872 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 151928531 0 0) 0x7fe2e00711c0 con 0x7fe2e0071ef0 2024-01-21T22:25:12.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.027+0000 7fe2dde00640 1 -- 172.21.15.138:0/3991943872 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2c0003520 con 0x7fe2e0071ef0 2024-01-21T22:25:12.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.031+0000 7fe2dde00640 1 -- 172.21.15.138:0/3991943872 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1379513625 0 0) 0x7fe2c0003520 con 0x7fe2e0071ef0 2024-01-21T22:25:12.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.031+0000 7fe2dde00640 1 -- 172.21.15.138:0/3991943872 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2e0153e40 msgr2=0x7fe2e006a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:12.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.031+0000 7fe2dde00640 1 --2- 172.21.15.138:0/3991943872 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2e0153e40 0x7fe2e006a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:12.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.031+0000 7fe2dde00640 1 -- 172.21.15.138:0/3991943872 >> v2:172.21.15.152:3300/0 conn(0x7fe2e00c6160 msgr2=0x7fe2e00c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:12.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.031+0000 7fe2dde00640 1 --2- 172.21.15.138:0/3991943872 >> v2:172.21.15.152:3300/0 conn(0x7fe2e00c6160 0x7fe2e00c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:12.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.031+0000 7fe2dde00640 1 -- 172.21.15.138:0/3991943872 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e015dba0 con 0x7fe2e0071ef0 2024-01-21T22:25:12.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.031+0000 7fe2dde00640 1 -- 172.21.15.138:0/3991943872 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1288616872 0 0) 0x7fe2d80019f0 con 0x7fe2e0071ef0 2024-01-21T22:25:12.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.031+0000 7fe2dde00640 1 -- 172.21.15.138:0/3991943872 >> v1:172.21.15.138:6789/0 conn(0x7fe2e0071ef0 legacy=0x7fe2e00c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:12.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.032+0000 7fe2dde00640 1 --2- 172.21.15.138:0/3991943872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2c0003970 0x7fe2c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:12.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.032+0000 7fe2dde00640 1 --2- 172.21.15.138:0/3991943872 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2c00047c0 0x7fe2c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:12.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.032+0000 7fe2dde00640 1 --2- 172.21.15.138:0/3991943872 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2e0153e40 0x7fe2c000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:12.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.032+0000 7fe2dde00640 1 -- 172.21.15.138:0/3991943872 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e00723a0 con 0x7fe2c0003970 2024-01-21T22:25:12.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.032+0000 7fe2dde00640 1 -- 172.21.15.138:0/3991943872 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e0154220 con 0x7fe2c00047c0 2024-01-21T22:25:12.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.032+0000 7fe2dde00640 1 -- 172.21.15.138:0/3991943872 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c00055d0 con 0x7fe2e0153e40 2024-01-21T22:25:12.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.036+0000 7fe2de800640 1 --2- 172.21.15.138:0/3991943872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2c0003970 0x7fe2c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:12.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.042+0000 7fe2de800640 1 -- 172.21.15.138:0/3991943872 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2e0153e40 msgr2=0x7fe2c000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:12.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.042+0000 7fe2de800640 1 --2- 172.21.15.138:0/3991943872 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2e0153e40 0x7fe2c000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:12.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.042+0000 7fe2de800640 1 -- 172.21.15.138:0/3991943872 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2c00047c0 msgr2=0x7fe2c0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:12.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.042+0000 7fe2de800640 1 --2- 172.21.15.138:0/3991943872 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2c00047c0 0x7fe2c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:12.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.042+0000 7fe2de800640 1 -- 172.21.15.138:0/3991943872 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c0003520 con 0x7fe2c0003970 2024-01-21T22:25:12.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.045+0000 7fe2de800640 1 --2- 172.21.15.138:0/3991943872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2c0003970 0x7fe2c0003d90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009860 tx=0x7fe2c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=430c768c424533d7 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:12.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.053+0000 7fe2dde00640 1 -- 172.21.15.138:0/3991943872 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0032600 con 0x7fe2c0003970 2024-01-21T22:25:12.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.053+0000 7fe2dde00640 1 -- 172.21.15.138:0/3991943872 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d002e050 con 0x7fe2c0003970 2024-01-21T22:25:12.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.053+0000 7fe2dde00640 1 -- 172.21.15.138:0/3991943872 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0032a00 con 0x7fe2c0003970 2024-01-21T22:25:12.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.054+0000 7fe2e5400640 1 -- 172.21.15.138:0/3991943872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2c0003970 msgr2=0x7fe2c0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:12.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.054+0000 7fe2e5400640 1 --2- 172.21.15.138:0/3991943872 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2c0003970 0x7fe2c0003d90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0009860 tx=0x7fe2c00057f0 comp rx=0 tx=0).stop 2024-01-21T22:25:12.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.054+0000 7fe2df200640 1 -- 172.21.15.138:0/3991943872 reap_dead start 2024-01-21T22:25:12.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.055+0000 7fe2e5400640 1 -- 172.21.15.138:0/3991943872 shutdown_connections 2024-01-21T22:25:12.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.055+0000 7fe2e5400640 1 -- 172.21.15.138:0/3991943872 >> 172.21.15.138:0/3991943872 conn(0x7fe2e00c8260 msgr2=0x7fe2e00c8670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:12.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.055+0000 7fe2e5400640 1 -- 172.21.15.138:0/3991943872 shutdown_connections 2024-01-21T22:25:12.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.055+0000 7fe2e5400640 1 -- 172.21.15.138:0/3991943872 wait complete. 2024-01-21T22:25:12.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.057+0000 7fe2e5400640 1 Processor -- start 2024-01-21T22:25:12.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.057+0000 7fe2e5400640 1 -- start start 2024-01-21T22:25:12.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.058+0000 7fe2e5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2c0003970 0x7fe2e00ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:12.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.058+0000 7fe2e5400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2e0154b80 0x7fe2e00be9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:12.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.058+0000 7fe2e5400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2e00beef0 0x7fe2e00bd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:12.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.058+0000 7fe2e5400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e014dcb0 con 0x7fe2c0003970 2024-01-21T22:25:12.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.058+0000 7fe2e5400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e00c8aa0 con 0x7fe2e00beef0 2024-01-21T22:25:12.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.058+0000 7fe2e5400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e00bf370 con 0x7fe2e0154b80 2024-01-21T22:25:12.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.059+0000 7fe2df200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2e00beef0 0x7fe2e00bd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:12.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.059+0000 7fe2d7e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2e0154b80 0x7fe2e00be9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:12.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.059+0000 7fe2d7e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2e0154b80 0x7fe2e00be9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:48406/0 (socket says 172.21.15.138:48406) 2024-01-21T22:25:12.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.059+0000 7fe2d7e00640 1 -- 172.21.15.138:0/639544742 learned_addr learned my addr 172.21.15.138:0/639544742 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:12.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.063+0000 7fe2de800640 1 --2- 172.21.15.138:0/639544742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2c0003970 0x7fe2e00ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:12.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.063+0000 7fe2d7e00640 1 -- 172.21.15.138:0/639544742 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2e00beef0 msgr2=0x7fe2e00bd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:12.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.063+0000 7fe2d7e00640 1 --2- 172.21.15.138:0/639544742 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2e00beef0 0x7fe2e00bd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:12.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.063+0000 7fe2d7e00640 1 -- 172.21.15.138:0/639544742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2c0003970 msgr2=0x7fe2e00ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:12.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.063+0000 7fe2d7e00640 1 --2- 172.21.15.138:0/639544742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2c0003970 0x7fe2e00ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:12.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.063+0000 7fe2d7e00640 1 -- 172.21.15.138:0/639544742 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e00caf70 con 0x7fe2e0154b80 2024-01-21T22:25:12.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.065+0000 7fe2d7e00640 1 --2- 172.21.15.138:0/639544742 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2e0154b80 0x7fe2e00be9b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800b500 tx=0x7fe2c800b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:12.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.069+0000 7fe2d6a00640 1 -- 172.21.15.138:0/639544742 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8010870 con 0x7fe2e0154b80 2024-01-21T22:25:12.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.069+0000 7fe2d6a00640 1 -- 172.21.15.138:0/639544742 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c80109d0 con 0x7fe2e0154b80 2024-01-21T22:25:12.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.069+0000 7fe2d6a00640 1 -- 172.21.15.138:0/639544742 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8010d80 con 0x7fe2e0154b80 2024-01-21T22:25:12.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.069+0000 7fe2e5400640 1 -- 172.21.15.138:0/639544742 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e00cb250 con 0x7fe2e0154b80 2024-01-21T22:25:12.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.069+0000 7fe2e5400640 1 -- 172.21.15.138:0/639544742 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e00c0ff0 con 0x7fe2e0154b80 2024-01-21T22:25:12.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.071+0000 7fe2e5400640 1 -- 172.21.15.138:0/639544742 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e004eb10 con 0x7fe2e0154b80 2024-01-21T22:25:12.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.089+0000 7fe2d6a00640 1 -- 172.21.15.138:0/639544742 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fe2c8023480 con 0x7fe2e0154b80 2024-01-21T22:25:12.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.091+0000 7fe2d6a00640 1 --2- 172.21.15.138:0/639544742 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe2b806b3b0 0x7fe2b806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:12.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.091+0000 7fe2d6a00640 1 -- 172.21.15.138:0/639544742 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2c8058910 con 0x7fe2e0154b80 2024-01-21T22:25:12.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.091+0000 7fe2de800640 1 --2- 172.21.15.138:0/639544742 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe2b806b3b0 0x7fe2b806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:12.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.092+0000 7fe2de800640 1 --2- 172.21.15.138:0/639544742 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe2b806b3b0 0x7fe2b806d870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0034000 tx=0x7fe2d0008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:12.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.323+0000 7fe2d6a00640 1 -- 172.21.15.138:0/639544742 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe2c8054c50 con 0x7fe2e0154b80 2024-01-21T22:25:12.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:12.570+0000 7fe2e5400640 1 -- 172.21.15.138:0/639544742 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fe2e00c6c50 con 0x7fe2e0154b80 2024-01-21T22:25:13.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.138+0000 7fe2d6a00640 1 -- 172.21.15.138:0/639544742 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v10) v1 ==== 115+0+0 (secure 0 0 0) 0x7fe2c8058270 con 0x7fe2e0154b80 2024-01-21T22:25:13.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.141+0000 7fe2e5400640 1 -- 172.21.15.138:0/639544742 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe2b806b3b0 msgr2=0x7fe2b806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:13.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.141+0000 7fe2e5400640 1 --2- 172.21.15.138:0/639544742 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe2b806b3b0 0x7fe2b806d870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe2d0034000 tx=0x7fe2d0008030 comp rx=0 tx=0).stop 2024-01-21T22:25:13.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.141+0000 7fe2e5400640 1 -- 172.21.15.138:0/639544742 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2e0154b80 msgr2=0x7fe2e00be9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:13.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.141+0000 7fe2e5400640 1 --2- 172.21.15.138:0/639544742 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2e0154b80 0x7fe2e00be9b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800b500 tx=0x7fe2c800b9c0 comp rx=0 tx=0).stop 2024-01-21T22:25:13.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.141+0000 7fe2e5400640 1 -- 172.21.15.138:0/639544742 shutdown_connections 2024-01-21T22:25:13.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.141+0000 7fe2e5400640 1 --2- 172.21.15.138:0/639544742 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe2b806b3b0 0x7fe2b806d870 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:13.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.141+0000 7fe2e5400640 1 --2- 172.21.15.138:0/639544742 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe2e00beef0 0x7fe2e00bd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:13.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.141+0000 7fe2e5400640 1 --2- 172.21.15.138:0/639544742 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe2e0154b80 0x7fe2e00be9b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:13.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.141+0000 7fe2e5400640 1 --2- 172.21.15.138:0/639544742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2c0003970 0x7fe2e00ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:13.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.141+0000 7fe2e5400640 1 -- 172.21.15.138:0/639544742 >> 172.21.15.138:0/639544742 conn(0x7fe2e00c8260 msgr2=0x7fe2e00c56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:13.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.142+0000 7fe2e5400640 1 -- 172.21.15.138:0/639544742 shutdown_connections 2024-01-21T22:25:13.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.142+0000 7fe2e5400640 1 -- 172.21.15.138:0/639544742 wait complete. 2024-01-21T22:25:13.168 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-01-21T22:25:13.168 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-21T22:25:13.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.370+0000 7fd5ada00640 1 Processor -- start 2024-01-21T22:25:13.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.370+0000 7fd5ada00640 1 -- start start 2024-01-21T22:25:13.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.371+0000 7fd5ada00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd5a8154ed0 0x7fd5a806a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:13.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.371+0000 7fd5ada00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5a80c9700 0x7fd5a80c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:13.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.371+0000 7fd5ada00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5a815be20 con 0x7fd5a806c2a0 2024-01-21T22:25:13.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.372+0000 7fd5ada00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a806b420 con 0x7fd5a8154ed0 2024-01-21T22:25:13.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.372+0000 7fd5ada00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a806ac30 con 0x7fd5a80c9700 2024-01-21T22:25:13.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.372+0000 7fd5a7400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fd5a806c2a0 0x7fd5a8157d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:60644/0 (socket says 172.21.15.138:60644) 2024-01-21T22:25:13.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.372+0000 7fd5a7400640 1 -- 172.21.15.138:0/2716368072 learned_addr learned my addr 172.21.15.138:0/2716368072 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:13.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.373+0000 7fd59de00640 1 --2- 172.21.15.138:0/2716368072 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5a80c9700 0x7fd5a80c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:13.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.373+0000 7fd5a6a00640 1 --2- 172.21.15.138:0/2716368072 >> v2:172.21.15.152:3300/0 conn(0x7fd5a8154ed0 0x7fd5a806a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:13.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.376+0000 7fd5a6a00640 1 -- 172.21.15.138:0/2716368072 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5a80c9700 msgr2=0x7fd5a80c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:13.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.376+0000 7fd5a6a00640 1 --2- 172.21.15.138:0/2716368072 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5a80c9700 0x7fd5a80c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:13.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.376+0000 7fd5a6a00640 1 -- 172.21.15.138:0/2716368072 >> v1:172.21.15.138:6789/0 conn(0x7fd5a806c2a0 legacy=0x7fd5a8157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:13.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.376+0000 7fd5a6a00640 1 -- 172.21.15.138:0/2716368072 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a815af30 con 0x7fd5a8154ed0 2024-01-21T22:25:13.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.379+0000 7fd5a6a00640 1 --2- 172.21.15.138:0/2716368072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5a8154ed0 0x7fd5a806a6f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd598009d90 tx=0x7fd59802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21df5dca1790206f server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:13.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.382+0000 7fd5a6000640 1 -- 172.21.15.138:0/2716368072 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59802cc50 con 0x7fd5a8154ed0 2024-01-21T22:25:13.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.382+0000 7fd5a6000640 1 -- 172.21.15.138:0/2716368072 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59802cdb0 con 0x7fd5a8154ed0 2024-01-21T22:25:13.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.382+0000 7fd5a6000640 1 -- 172.21.15.138:0/2716368072 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598034660 con 0x7fd5a8154ed0 2024-01-21T22:25:13.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.383+0000 7fd5ada00640 1 -- 172.21.15.138:0/2716368072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5a8154ed0 msgr2=0x7fd5a806a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:13.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.383+0000 7fd5ada00640 1 --2- 172.21.15.138:0/2716368072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5a8154ed0 0x7fd5a806a6f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd598009d90 tx=0x7fd59802bea0 comp rx=0 tx=0).stop 2024-01-21T22:25:13.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.383+0000 7fd5ada00640 1 -- 172.21.15.138:0/2716368072 shutdown_connections 2024-01-21T22:25:13.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.383+0000 7fd5ada00640 1 --2- 172.21.15.138:0/2716368072 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5a80c9700 0x7fd5a80c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:13.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.383+0000 7fd5ada00640 1 --2- 172.21.15.138:0/2716368072 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5a8154ed0 0x7fd5a806a6f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:13.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.383+0000 7fd5ada00640 1 -- 172.21.15.138:0/2716368072 >> 172.21.15.138:0/2716368072 conn(0x7fd5a8066a90 msgr2=0x7fd5a80722f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:13.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.383+0000 7fd5ada00640 1 -- 172.21.15.138:0/2716368072 shutdown_connections 2024-01-21T22:25:13.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.383+0000 7fd5ada00640 1 -- 172.21.15.138:0/2716368072 wait complete. 2024-01-21T22:25:13.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.386+0000 7fd5ada00640 1 Processor -- start 2024-01-21T22:25:13.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.386+0000 7fd5ada00640 1 -- start start 2024-01-21T22:25:13.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.387+0000 7fd5ada00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5a806c2a0 0x7fd5a81e18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:13.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.387+0000 7fd5ada00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5a80c9700 0x7fd5a815a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:13.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.387+0000 7fd5ada00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5a8154ed0 0x7fd5a814efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:13.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.387+0000 7fd5ada00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a8151690 con 0x7fd5a80c9700 2024-01-21T22:25:13.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.387+0000 7fd5ada00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a81e1df0 con 0x7fd5a806c2a0 2024-01-21T22:25:13.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.387+0000 7fd5ada00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a814eb80 con 0x7fd5a8154ed0 2024-01-21T22:25:13.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.388+0000 7fd5a7400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5a8154ed0 0x7fd5a814efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:13.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.388+0000 7fd5a6a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5a806c2a0 0x7fd5a81e18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:13.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.388+0000 7fd5a7400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5a8154ed0 0x7fd5a814efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:48422/0 (socket says 172.21.15.138:48422) 2024-01-21T22:25:13.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.388+0000 7fd5a7400640 1 -- 172.21.15.138:0/2478574227 learned_addr learned my addr 172.21.15.138:0/2478574227 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:13.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.392+0000 7fd5a7400640 1 -- 172.21.15.138:0/2478574227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5a806c2a0 msgr2=0x7fd5a81e18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:13.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.392+0000 7fd5a7400640 1 --2- 172.21.15.138:0/2478574227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5a806c2a0 0x7fd5a81e18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:13.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.392+0000 7fd5a7400640 1 -- 172.21.15.138:0/2478574227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5a80c9700 msgr2=0x7fd5a815a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:13.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.392+0000 7fd5a7400640 1 --2- 172.21.15.138:0/2478574227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5a80c9700 0x7fd5a815a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:13.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.392+0000 7fd5a7400640 1 -- 172.21.15.138:0/2478574227 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd598009a70 con 0x7fd5a8154ed0 2024-01-21T22:25:13.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.394+0000 7fd5a7400640 1 --2- 172.21.15.138:0/2478574227 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5a8154ed0 0x7fd5a814efd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000c3e0 tx=0x7fd5a000c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:13.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.396+0000 7fd59f400640 1 -- 172.21.15.138:0/2478574227 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a0005ce0 con 0x7fd5a8154ed0 2024-01-21T22:25:13.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.396+0000 7fd5ada00640 1 -- 172.21.15.138:0/2478574227 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a81513b0 con 0x7fd5a8154ed0 2024-01-21T22:25:13.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.396+0000 7fd5ada00640 1 -- 172.21.15.138:0/2478574227 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a81ee1e0 con 0x7fd5a8154ed0 2024-01-21T22:25:13.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.397+0000 7fd59f400640 1 -- 172.21.15.138:0/2478574227 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a0004af0 con 0x7fd5a8154ed0 2024-01-21T22:25:13.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.398+0000 7fd59f400640 1 -- 172.21.15.138:0/2478574227 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a00072d0 con 0x7fd5a8154ed0 2024-01-21T22:25:13.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.398+0000 7fd5ada00640 1 -- 172.21.15.138:0/2478574227 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5a81edd60 con 0x7fd5a8154ed0 2024-01-21T22:25:13.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.420+0000 7fd59f400640 1 -- 172.21.15.138:0/2478574227 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fd5a00074f0 con 0x7fd5a8154ed0 2024-01-21T22:25:13.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.422+0000 7fd59f400640 1 --2- 172.21.15.138:0/2478574227 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd58006b330 0x7fd58006d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:13.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.422+0000 7fd5a6a00640 1 --2- 172.21.15.138:0/2478574227 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd58006b330 0x7fd58006d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:13.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.423+0000 7fd5a6a00640 1 --2- 172.21.15.138:0/2478574227 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd58006b330 0x7fd58006d7f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd598036000 tx=0x7fd598030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:13.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.645+0000 7fd59f400640 1 -- 172.21.15.138:0/2478574227 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5a0005e40 con 0x7fd5a8154ed0 2024-01-21T22:25:13.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.656+0000 7fd59f400640 1 -- 172.21.15.138:0/2478574227 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd5a00547f0 con 0x7fd5a8154ed0 2024-01-21T22:25:13.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.891+0000 7fd5ada00640 1 -- 172.21.15.138:0/2478574227 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5a81edd60 con 0x7fd5a8154ed0 2024-01-21T22:25:13.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.913+0000 7fd59f400640 1 -- 172.21.15.138:0/2478574227 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+6530 (secure 0 0 0) 0x7fd5a81edd60 con 0x7fd5a8154ed0 2024-01-21T22:25:13.916 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:25:13.916 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/3112363661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":3112363661},{"type":"v1","addr":"172.21.15.152:6837","nonce":3112363661}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4223,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/3222259940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":3222259940},{"type":"v1","addr":"172.21.15.152:6839","nonce":3222259940}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/909784319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":909784319},{"type":"v1","addr":"172.21.15.138:6833","nonce":909784319}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4229,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3318550662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3318550662},{"type":"v1","addr":"172.21.15.138:6835","nonce":3318550662}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2986531362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2986531362},{"type":"v1","addr":"172.21.15.138:6837","nonce":2986531362}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-21T22:25:04.967250+0000","modified":"2024-01-21T22:25:12.598970+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":5,"state":"up:active","state_seq":5,"addr":"172.21.15.152:6835/2663179600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2663179600},{"type":"v1","addr":"172.21.15.152:6835","nonce":2663179600}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-21T22:25:13.916 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 10 2024-01-21T22:25:13.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.916+0000 7fd5ada00640 1 -- 172.21.15.138:0/2478574227 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd58006b330 msgr2=0x7fd58006d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:13.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.916+0000 7fd5ada00640 1 --2- 172.21.15.138:0/2478574227 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd58006b330 0x7fd58006d7f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd598036000 tx=0x7fd598030000 comp rx=0 tx=0).stop 2024-01-21T22:25:13.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.916+0000 7fd5ada00640 1 -- 172.21.15.138:0/2478574227 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5a8154ed0 msgr2=0x7fd5a814efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:13.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.916+0000 7fd5ada00640 1 --2- 172.21.15.138:0/2478574227 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5a8154ed0 0x7fd5a814efd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000c3e0 tx=0x7fd5a000c8a0 comp rx=0 tx=0).stop 2024-01-21T22:25:13.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.916+0000 7fd5ada00640 1 -- 172.21.15.138:0/2478574227 shutdown_connections 2024-01-21T22:25:13.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.916+0000 7fd5ada00640 1 --2- 172.21.15.138:0/2478574227 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd58006b330 0x7fd58006d7f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:13.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.916+0000 7fd5ada00640 1 --2- 172.21.15.138:0/2478574227 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd5a8154ed0 0x7fd5a814efd0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:13.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.916+0000 7fd5ada00640 1 --2- 172.21.15.138:0/2478574227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5a80c9700 0x7fd5a815a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:13.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.916+0000 7fd5ada00640 1 --2- 172.21.15.138:0/2478574227 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd5a806c2a0 0x7fd5a81e18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:13.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.916+0000 7fd5ada00640 1 -- 172.21.15.138:0/2478574227 >> 172.21.15.138:0/2478574227 conn(0x7fd5a8066a90 msgr2=0x7fd5a8152c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:13.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.917+0000 7fd5ada00640 1 -- 172.21.15.138:0/2478574227 shutdown_connections 2024-01-21T22:25:13.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:13.917+0000 7fd5ada00640 1 -- 172.21.15.138:0/2478574227 wait complete. 2024-01-21T22:25:13.939 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-21T22:25:14.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.138+0000 7f6aef000640 1 Processor -- start 2024-01-21T22:25:14.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.138+0000 7f6aef000640 1 -- start start 2024-01-21T22:25:14.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.139+0000 7f6aef000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f6ae80c4370 0x7f6ae815a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:14.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.139+0000 7f6aef000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae80c47e0 con 0x7f6ae8153c70 2024-01-21T22:25:14.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.139+0000 7f6aef000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae8159a90 con 0x7f6ae815a7b0 2024-01-21T22:25:14.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.139+0000 7f6aef000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ae8159e50 con 0x7f6ae80c4370 2024-01-21T22:25:14.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.158+0000 7f6aed000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f6ae815a7b0 0x7f6ae80cc260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:58926/0 (socket says 172.21.15.138:58926) 2024-01-21T22:25:14.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.158+0000 7f6aed000640 1 -- 172.21.15.138:0/2715066209 learned_addr learned my addr 172.21.15.138:0/2715066209 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:14.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.216+0000 7f6ae7400640 1 -- 172.21.15.138:0/2715066209 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2269663057 0 0) 0x7f6ae8159a90 con 0x7f6ae815a7b0 2024-01-21T22:25:14.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.216+0000 7f6ae7400640 1 -- 172.21.15.138:0/2715066209 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ac8003520 con 0x7f6ae815a7b0 2024-01-21T22:25:14.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.220+0000 7f6ae7400640 1 -- 172.21.15.138:0/2715066209 <== mon.1 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2181046295 0 0) 0x7f6ac8003520 con 0x7f6ae815a7b0 2024-01-21T22:25:14.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.220+0000 7f6ae7400640 1 -- 172.21.15.138:0/2715066209 >> v2:172.21.15.138:3300/0 conn(0x7f6ae80c4370 msgr2=0x7f6ae815a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:14.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.220+0000 7f6ae7400640 1 --2- 172.21.15.138:0/2715066209 >> v2:172.21.15.138:3300/0 conn(0x7f6ae80c4370 0x7f6ae815a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:14.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.220+0000 7f6ae7400640 1 -- 172.21.15.138:0/2715066209 >> v1:172.21.15.138:6789/0 conn(0x7f6ae8153c70 legacy=0x7f6ae80c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:14.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.220+0000 7f6ae7400640 1 -- 172.21.15.138:0/2715066209 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae815d5e0 con 0x7f6ae815a7b0 2024-01-21T22:25:14.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.220+0000 7f6ae7400640 1 -- 172.21.15.138:0/2715066209 <== mon.1 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1288616872 0 0) 0x7f6ae00019f0 con 0x7f6ae815a7b0 2024-01-21T22:25:14.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.220+0000 7f6ae7400640 1 -- 172.21.15.138:0/2715066209 >> v1:172.21.15.152:6789/0 conn(0x7f6ae815a7b0 legacy=0x7f6ae80cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:14.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.220+0000 7f6ae7400640 1 --2- 172.21.15.138:0/2715066209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8003970 0x7f6ac8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:14.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.221+0000 7f6ae7400640 1 --2- 172.21.15.138:0/2715066209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ac80047c0 0x7f6ac8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:14.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.221+0000 7f6ae7400640 1 --2- 172.21.15.138:0/2715066209 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6ac8005380 0x7f6ac800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:14.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.221+0000 7f6ae7400640 1 -- 172.21.15.138:0/2715066209 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae8159e50 con 0x7f6ac80047c0 2024-01-21T22:25:14.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.221+0000 7f6ae7400640 1 -- 172.21.15.138:0/2715066209 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac800c240 con 0x7f6ac8003970 2024-01-21T22:25:14.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.221+0000 7f6ae7400640 1 -- 172.21.15.138:0/2715066209 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac800c400 con 0x7f6ac8005380 2024-01-21T22:25:14.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.222+0000 7f6aed000640 1 --2- 172.21.15.138:0/2715066209 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6ac8005380 0x7f6ac800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:14.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.223+0000 7f6ae7e00640 1 --2- 172.21.15.138:0/2715066209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8003970 0x7f6ac8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:14.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.225+0000 7f6aed000640 1 -- 172.21.15.138:0/2715066209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8003970 msgr2=0x7f6ac8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:14.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.225+0000 7f6aed000640 1 --2- 172.21.15.138:0/2715066209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8003970 0x7f6ac8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:14.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.225+0000 7f6aed000640 1 -- 172.21.15.138:0/2715066209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ac80047c0 msgr2=0x7f6ac8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:14.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.225+0000 7f6aed000640 1 --2- 172.21.15.138:0/2715066209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ac80047c0 0x7f6ac8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:14.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.225+0000 7f6aed000640 1 -- 172.21.15.138:0/2715066209 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac8003520 con 0x7f6ac8005380 2024-01-21T22:25:14.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.227+0000 7f6aed000640 1 -- 172.21.15.138:0/2715066209 reap_dead start 2024-01-21T22:25:14.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.229+0000 7f6aed000640 1 --2- 172.21.15.138:0/2715066209 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6ac8005380 0x7f6ac800b980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000a800 tx=0x7f6ac80057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=985e21c7b68183a6 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:14.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.233+0000 7f6ae7400640 1 -- 172.21.15.138:0/2715066209 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00072d0 con 0x7f6ac8005380 2024-01-21T22:25:14.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.233+0000 7f6ae7400640 1 -- 172.21.15.138:0/2715066209 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae002f070 con 0x7f6ac8005380 2024-01-21T22:25:14.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.233+0000 7f6ae7400640 1 -- 172.21.15.138:0/2715066209 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0007670 con 0x7f6ac8005380 2024-01-21T22:25:14.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.235+0000 7f6aef000640 1 -- 172.21.15.138:0/2715066209 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6ac8005380 msgr2=0x7f6ac800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:14.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.235+0000 7f6aef000640 1 --2- 172.21.15.138:0/2715066209 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6ac8005380 0x7f6ac800b980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000a800 tx=0x7f6ac80057f0 comp rx=0 tx=0).stop 2024-01-21T22:25:14.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.235+0000 7f6aef000640 1 -- 172.21.15.138:0/2715066209 shutdown_connections 2024-01-21T22:25:14.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.235+0000 7f6aef000640 1 --2- 172.21.15.138:0/2715066209 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6ac8005380 0x7f6ac800b980 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:14.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.235+0000 7f6aef000640 1 -- 172.21.15.138:0/2715066209 >> 172.21.15.138:0/2715066209 conn(0x7f6ae8066820 msgr2=0x7f6ae8066c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:14.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.235+0000 7f6aef000640 1 -- 172.21.15.138:0/2715066209 shutdown_connections 2024-01-21T22:25:14.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.235+0000 7f6aef000640 1 -- 172.21.15.138:0/2715066209 wait complete. 2024-01-21T22:25:14.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.238+0000 7f6aef000640 1 Processor -- start 2024-01-21T22:25:14.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.239+0000 7f6aef000640 1 -- start start 2024-01-21T22:25:14.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.240+0000 7f6aef000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8005380 0x7f6ae8151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:14.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.240+0000 7f6aef000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6ae8150180 0x7f6ae814a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:14.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.240+0000 7f6aef000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ae81f3040 0x7f6ae81f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:14.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.240+0000 7f6aef000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae814d670 con 0x7f6ae81f3040 2024-01-21T22:25:14.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.240+0000 7f6aef000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae814ca60 con 0x7f6ac8005380 2024-01-21T22:25:14.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.240+0000 7f6aef000640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae81f3a00 con 0x7f6ae8150180 2024-01-21T22:25:14.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.241+0000 7f6ae7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8005380 0x7f6ae8151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:14.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.241+0000 7f6aed000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ae81f3040 0x7f6ae81f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:14.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.241+0000 7f6adfe00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6ae8150180 0x7f6ae814a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:14.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.241+0000 7f6adfe00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6ae8150180 0x7f6ae814a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:48440/0 (socket says 172.21.15.138:48440) 2024-01-21T22:25:14.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.241+0000 7f6ae7e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8005380 0x7f6ae8151490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:47814/0 (socket says 172.21.15.138:47814) 2024-01-21T22:25:14.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.241+0000 7f6adfe00640 1 -- 172.21.15.138:0/688556020 learned_addr learned my addr 172.21.15.138:0/688556020 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:14.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.245+0000 7f6ae7e00640 1 -- 172.21.15.138:0/688556020 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6ae8150180 msgr2=0x7f6ae814a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:14.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.245+0000 7f6ae7e00640 1 --2- 172.21.15.138:0/688556020 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6ae8150180 0x7f6ae814a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:14.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.245+0000 7f6ae7e00640 1 -- 172.21.15.138:0/688556020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ae81f3040 msgr2=0x7f6ae81f34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:14.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.245+0000 7f6ae7e00640 1 --2- 172.21.15.138:0/688556020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ae81f3040 0x7f6ae81f34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:14.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.245+0000 7f6ae7e00640 1 -- 172.21.15.138:0/688556020 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae814b290 con 0x7f6ac8005380 2024-01-21T22:25:14.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.245+0000 7f6adfe00640 1 --2- 172.21.15.138:0/688556020 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6ae8150180 0x7f6ae814a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-21T22:25:14.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.248+0000 7f6ae7e00640 1 --2- 172.21.15.138:0/688556020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8005380 0x7f6ae8151490 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800f580 tx=0x7f6ad800fa40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:14.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.251+0000 7f6ae4c00640 1 -- 172.21.15.138:0/688556020 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad80053a0 con 0x7f6ac8005380 2024-01-21T22:25:14.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.251+0000 7f6ae4c00640 1 -- 172.21.15.138:0/688556020 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad80035f0 con 0x7f6ac8005380 2024-01-21T22:25:14.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.251+0000 7f6ae4c00640 1 -- 172.21.15.138:0/688556020 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad80039a0 con 0x7f6ac8005380 2024-01-21T22:25:14.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.252+0000 7f6aef000640 1 -- 172.21.15.138:0/688556020 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae814ccd0 con 0x7f6ac8005380 2024-01-21T22:25:14.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.252+0000 7f6aef000640 1 -- 172.21.15.138:0/688556020 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae814d290 con 0x7f6ac8005380 2024-01-21T22:25:14.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.253+0000 7f6aef000640 1 -- 172.21.15.138:0/688556020 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae804eaa0 con 0x7f6ac8005380 2024-01-21T22:25:14.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.273+0000 7f6ae4c00640 1 -- 172.21.15.138:0/688556020 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f6ad8003b00 con 0x7f6ac8005380 2024-01-21T22:25:14.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.274+0000 7f6ae4c00640 1 --2- 172.21.15.138:0/688556020 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6ac006b3b0 0x7f6ac006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:14.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.274+0000 7f6adfe00640 1 --2- 172.21.15.138:0/688556020 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6ac006b3b0 0x7f6ac006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:14.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.275+0000 7f6ae4c00640 1 -- 172.21.15.138:0/688556020 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ad8058d30 con 0x7f6ac8005380 2024-01-21T22:25:14.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.275+0000 7f6adfe00640 1 --2- 172.21.15.138:0/688556020 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6ac006b3b0 0x7f6ac006d870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80b5450 tx=0x7f6ad0009240 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:14.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.512+0000 7f6ae4c00640 1 -- 172.21.15.138:0/688556020 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6ad8055070 con 0x7f6ac8005380 2024-01-21T22:25:14.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.716+0000 7f6aef000640 1 -- 172.21.15.138:0/688556020 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ae815a1e0 con 0x7f6ac8005380 2024-01-21T22:25:14.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.760+0000 7f6ae4c00640 1 -- 172.21.15.138:0/688556020 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16522 (secure 0 0 0) 0x7f6ad8058690 con 0x7f6ac8005380 2024-01-21T22:25:14.762 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:25:14.762 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":23,"fsid":"d488d446-149f-48d0-89ca-b2830ae72846","created":"2024-01-21T22:24:28.581932+0000","modified":"2024-01-21T22:25:06.475790+0000","last_up_change":"2024-01-21T22:24:47.145799+0000","last_in_change":"2024-01-21T22:24:37.672807+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-21T22:24:49.586178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-21T22:25:02.044454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-21T22:25:03.361765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d812b2d6-c8a3-4015-bfdb-5766b25eae5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6825","nonce":2849203277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6827","nonce":2849203277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6831","nonce":2849203277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6829","nonce":2849203277}]},"public_addr":"172.21.15.138:6825/2849203277","cluster_addr":"172.21.15.138:6827/2849203277","heartbeat_back_addr":"172.21.15.138:6831/2849203277","heartbeat_front_addr":"172.21.15.138:6829/2849203277","state":["exists","up"]},{"osd":1,"uuid":"d8d875df-c840-46cc-b6c0-a847f9c93999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6817","nonce":174521731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6819","nonce":174521731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6823","nonce":174521731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6821","nonce":174521731}]},"public_addr":"172.21.15.138:6817/174521731","cluster_addr":"172.21.15.138:6819/174521731","heartbeat_back_addr":"172.21.15.138:6823/174521731","heartbeat_front_addr":"172.21.15.138:6821/174521731","state":["exists","up"]},{"osd":2,"uuid":"4bd94373-76ca-47a6-870b-16d96d7c9e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6801","nonce":1365838748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6803","nonce":1365838748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6809","nonce":1365838748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6805","nonce":1365838748}]},"public_addr":"172.21.15.138:6801/1365838748","cluster_addr":"172.21.15.138:6803/1365838748","heartbeat_back_addr":"172.21.15.138:6809/1365838748","heartbeat_front_addr":"172.21.15.138:6805/1365838748","state":["exists","up"]},{"osd":3,"uuid":"07267087-d4d2-4e88-ab9b-bb87f6dd061a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6808","nonce":1841873885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6811","nonce":1841873885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6815","nonce":1841873885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6813","nonce":1841873885}]},"public_addr":"172.21.15.138:6808/1841873885","cluster_addr":"172.21.15.138:6811/1841873885","heartbeat_back_addr":"172.21.15.138:6815/1841873885","heartbeat_front_addr":"172.21.15.138:6813/1841873885","state":["exists","up"]},{"osd":4,"uuid":"b47ef758-d783-4168-a9a5-11db3de4ca12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6827","nonce":3960812298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6829","nonce":3960812298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6833","nonce":3960812298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6831","nonce":3960812298}]},"public_addr":"172.21.15.152:6827/3960812298","cluster_addr":"172.21.15.152:6829/3960812298","heartbeat_back_addr":"172.21.15.152:6833/3960812298","heartbeat_front_addr":"172.21.15.152:6831/3960812298","state":["exists","up"]},{"osd":5,"uuid":"0ea5d43e-61d8-4823-815f-9e42c62fae97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6819","nonce":4095234314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6821","nonce":4095234314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6825","nonce":4095234314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6823","nonce":4095234314}]},"public_addr":"172.21.15.152:6819/4095234314","cluster_addr":"172.21.15.152:6821/4095234314","heartbeat_back_addr":"172.21.15.152:6825/4095234314","heartbeat_front_addr":"172.21.15.152:6823/4095234314","state":["exists","up"]},{"osd":6,"uuid":"59ffac36-e2e0-4693-ac82-ad5eccf6224d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6803","nonce":223848127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6805","nonce":223848127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6809","nonce":223848127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6807","nonce":223848127}]},"public_addr":"172.21.15.152:6803/223848127","cluster_addr":"172.21.15.152:6805/223848127","heartbeat_back_addr":"172.21.15.152:6809/223848127","heartbeat_front_addr":"172.21.15.152:6807/223848127","state":["exists","up"]},{"osd":7,"uuid":"4f892c7d-32a5-4960-bf88-4ed7c863cd0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6811","nonce":52509095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6813","nonce":52509095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6817","nonce":52509095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6815","nonce":52509095}]},"public_addr":"172.21.15.152:6811/52509095","cluster_addr":"172.21.15.152:6813/52509095","heartbeat_back_addr":"172.21.15.152:6817/52509095","heartbeat_front_addr":"172.21.15.152:6815/52509095","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.434460+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.374709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.613134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.298148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.242144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.198102+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.093385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.197180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-21T22:25:14.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.762+0000 7f6aef000640 1 -- 172.21.15.138:0/688556020 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6ac006b3b0 msgr2=0x7f6ac006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:14.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.762+0000 7f6aef000640 1 --2- 172.21.15.138:0/688556020 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6ac006b3b0 0x7f6ac006d870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80b5450 tx=0x7f6ad0009240 comp rx=0 tx=0).stop 2024-01-21T22:25:14.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.762+0000 7f6aef000640 1 -- 172.21.15.138:0/688556020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8005380 msgr2=0x7f6ae8151490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:14.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.762+0000 7f6aef000640 1 --2- 172.21.15.138:0/688556020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8005380 0x7f6ae8151490 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6ad800f580 tx=0x7f6ad800fa40 comp rx=0 tx=0).stop 2024-01-21T22:25:14.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.763+0000 7f6aef000640 1 -- 172.21.15.138:0/688556020 shutdown_connections 2024-01-21T22:25:14.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.763+0000 7f6aef000640 1 --2- 172.21.15.138:0/688556020 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6ac006b3b0 0x7f6ac006d870 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:14.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.763+0000 7f6aef000640 1 --2- 172.21.15.138:0/688556020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ae81f3040 0x7f6ae81f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:14.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.763+0000 7f6aef000640 1 --2- 172.21.15.138:0/688556020 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6ae8150180 0x7f6ae814a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:14.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.763+0000 7f6aef000640 1 --2- 172.21.15.138:0/688556020 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6ac8005380 0x7f6ae8151490 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:14.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.763+0000 7f6aef000640 1 -- 172.21.15.138:0/688556020 >> 172.21.15.138:0/688556020 conn(0x7f6ae8066820 msgr2=0x7f6ae80cd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:14.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.763+0000 7f6aef000640 1 -- 172.21.15.138:0/688556020 shutdown_connections 2024-01-21T22:25:14.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.763+0000 7f6aef000640 1 -- 172.21.15.138:0/688556020 wait complete. 2024-01-21T22:25:14.785 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-01-21T22:25:14.785 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-21T22:25:14.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.994+0000 7f6243c00640 1 Processor -- start 2024-01-21T22:25:14.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.994+0000 7f6243c00640 1 -- start start 2024-01-21T22:25:14.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.996+0000 7f6243c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f623c0cd4f0 0x7f623c1585c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:14.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.996+0000 7f6243c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f623c06f750 0x7f623c158b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:14.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.996+0000 7f6243c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f623c0b2900 con 0x7f623c0cd8d0 2024-01-21T22:25:14.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.996+0000 7f6243c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f623c0b1380 con 0x7f623c0cd4f0 2024-01-21T22:25:14.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.996+0000 7f6243c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f623c0b0cb0 con 0x7f623c06f750 2024-01-21T22:25:14.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.997+0000 7f6241200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f623c0cd4f0 0x7f623c1585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:14.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.997+0000 7f6241c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f623c06f750 0x7f623c158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:14.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.997+0000 7f6233c00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f623c0cd8d0 0x7f623c159580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:60660/0 (socket says 172.21.15.138:60660) 2024-01-21T22:25:14.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.997+0000 7f6233c00640 1 -- 172.21.15.138:0/1167870796 learned_addr learned my addr 172.21.15.138:0/1167870796 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:15.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.999+0000 7f6241c00640 1 -- 172.21.15.138:0/1167870796 >> v2:172.21.15.138:3300/0 conn(0x7f623c0cd4f0 msgr2=0x7f623c1585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:15.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.999+0000 7f6241c00640 1 --2- 172.21.15.138:0/1167870796 >> v2:172.21.15.138:3300/0 conn(0x7f623c0cd4f0 0x7f623c1585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:15.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.999+0000 7f6241c00640 1 -- 172.21.15.138:0/1167870796 >> v1:172.21.15.138:6789/0 conn(0x7f623c0cd8d0 legacy=0x7f623c159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:15.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:14.999+0000 7f6241c00640 1 -- 172.21.15.138:0/1167870796 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c0b2340 con 0x7f623c06f750 2024-01-21T22:25:15.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.002+0000 7f6241c00640 1 --2- 172.21.15.138:0/1167870796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f623c06f750 0x7f623c158b00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f623c0c6e80 tx=0x7f623402bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1e9abe600862cc3 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:15.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.005+0000 7f623be00640 1 -- 172.21.15.138:0/1167870796 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623402cc80 con 0x7f623c06f750 2024-01-21T22:25:15.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.005+0000 7f623be00640 1 -- 172.21.15.138:0/1167870796 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623402cde0 con 0x7f623c06f750 2024-01-21T22:25:15.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.005+0000 7f623be00640 1 -- 172.21.15.138:0/1167870796 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6234034660 con 0x7f623c06f750 2024-01-21T22:25:15.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.006+0000 7f6243c00640 1 -- 172.21.15.138:0/1167870796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f623c06f750 msgr2=0x7f623c158b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:15.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.006+0000 7f6243c00640 1 --2- 172.21.15.138:0/1167870796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f623c06f750 0x7f623c158b00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f623c0c6e80 tx=0x7f623402bed0 comp rx=0 tx=0).stop 2024-01-21T22:25:15.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.006+0000 7f6243c00640 1 -- 172.21.15.138:0/1167870796 shutdown_connections 2024-01-21T22:25:15.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.006+0000 7f6243c00640 1 --2- 172.21.15.138:0/1167870796 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f623c06f750 0x7f623c158b00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:15.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.006+0000 7f6243c00640 1 --2- 172.21.15.138:0/1167870796 >> v2:172.21.15.138:3300/0 conn(0x7f623c0cd4f0 0x7f623c1585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:15.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.006+0000 7f6243c00640 1 -- 172.21.15.138:0/1167870796 >> 172.21.15.138:0/1167870796 conn(0x7f623c06ad20 msgr2=0x7f623c066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:15.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.006+0000 7f6243c00640 1 -- 172.21.15.138:0/1167870796 shutdown_connections 2024-01-21T22:25:15.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.006+0000 7f6243c00640 1 -- 172.21.15.138:0/1167870796 wait complete. 2024-01-21T22:25:15.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.008+0000 7f6243c00640 1 Processor -- start 2024-01-21T22:25:15.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.009+0000 7f6243c00640 1 -- start start 2024-01-21T22:25:15.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.009+0000 7f6243c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f623c06f750 0x7f623c0ca5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:15.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.009+0000 7f6243c00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f623c0cd4f0 0x7f623c151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:15.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.010+0000 7f6243c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f623c0cd8d0 0x7f623c151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:15.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.010+0000 7f6243c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c155e40 con 0x7f623c06f750 2024-01-21T22:25:15.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.010+0000 7f6243c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c1511c0 con 0x7f623c0cd8d0 2024-01-21T22:25:15.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.010+0000 7f6243c00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623c14fd50 con 0x7f623c0cd4f0 2024-01-21T22:25:15.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.010+0000 7f6241200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f623c06f750 0x7f623c0ca5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:15.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.010+0000 7f6241c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f623c0cd8d0 0x7f623c151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:15.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.010+0000 7f6233c00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f623c0cd4f0 0x7f623c151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:15.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.010+0000 7f6241200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f623c06f750 0x7f623c0ca5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43556/0 (socket says 172.21.15.138:43556) 2024-01-21T22:25:15.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.010+0000 7f6241200640 1 -- 172.21.15.138:0/4101479849 learned_addr learned my addr 172.21.15.138:0/4101479849 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:15.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.014+0000 7f6241c00640 1 -- 172.21.15.138:0/4101479849 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f623c0cd4f0 msgr2=0x7f623c151650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:15.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.014+0000 7f6241c00640 1 --2- 172.21.15.138:0/4101479849 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f623c0cd4f0 0x7f623c151650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:15.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.014+0000 7f6241c00640 1 -- 172.21.15.138:0/4101479849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f623c06f750 msgr2=0x7f623c0ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:15.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.014+0000 7f6241c00640 1 --2- 172.21.15.138:0/4101479849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f623c06f750 0x7f623c0ca5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:15.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.014+0000 7f6241c00640 1 -- 172.21.15.138:0/4101479849 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6234009a70 con 0x7f623c0cd8d0 2024-01-21T22:25:15.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.017+0000 7f6241c00640 1 --2- 172.21.15.138:0/4101479849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f623c0cd8d0 0x7f623c151b90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6234033000 tx=0x7f6234004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:15.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.020+0000 7f6239600640 1 -- 172.21.15.138:0/4101479849 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623403e610 con 0x7f623c0cd8d0 2024-01-21T22:25:15.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.020+0000 7f6239600640 1 -- 172.21.15.138:0/4101479849 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623403e770 con 0x7f623c0cd8d0 2024-01-21T22:25:15.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.020+0000 7f6239600640 1 -- 172.21.15.138:0/4101479849 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623403eb20 con 0x7f623c0cd8d0 2024-01-21T22:25:15.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.020+0000 7f6243c00640 1 -- 172.21.15.138:0/4101479849 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623c14fa70 con 0x7f623c0cd8d0 2024-01-21T22:25:15.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.020+0000 7f6243c00640 1 -- 172.21.15.138:0/4101479849 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623c150e50 con 0x7f623c0cd8d0 2024-01-21T22:25:15.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.022+0000 7f6243c00640 1 -- 172.21.15.138:0/4101479849 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623c15b450 con 0x7f623c0cd8d0 2024-01-21T22:25:15.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.275+0000 7f6239600640 1 -- 172.21.15.138:0/4101479849 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f623403b050 con 0x7f623c0cd8d0 2024-01-21T22:25:15.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.277+0000 7f6239600640 1 --2- 172.21.15.138:0/4101479849 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f621406b380 0x7f621406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:15.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.277+0000 7f6239600640 1 -- 172.21.15.138:0/4101479849 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62340ac610 con 0x7f623c0cd8d0 2024-01-21T22:25:15.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.278+0000 7f6241200640 1 --2- 172.21.15.138:0/4101479849 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f621406b380 0x7f621406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:15.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.279+0000 7f6241200640 1 --2- 172.21.15.138:0/4101479849 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f621406b380 0x7f621406d840 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f623c154890 tx=0x7f622c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:15.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.286+0000 7f6239600640 1 -- 172.21.15.138:0/4101479849 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f62340775e0 con 0x7f623c0cd8d0 2024-01-21T22:25:15.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.506+0000 7f6243c00640 1 -- 172.21.15.138:0/4101479849 --> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f623c0b0940 con 0x7f621406b380 2024-01-21T22:25:15.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.515+0000 7f6239600640 1 -- 172.21.15.138:0/4101479849 <== mgr.4111 v2:172.21.15.152:6800/314625564 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264526 (secure 0 0 0) 0x7f622c011af0 con 0x7f621406b380 2024-01-21T22:25:15.520 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:25:15.523 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-01-21T22:25:15.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.522+0000 7f6243c00640 1 -- 172.21.15.138:0/4101479849 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f621406b380 msgr2=0x7f621406d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:15.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.522+0000 7f6243c00640 1 --2- 172.21.15.138:0/4101479849 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f621406b380 0x7f621406d840 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f623c154890 tx=0x7f622c009190 comp rx=0 tx=0).stop 2024-01-21T22:25:15.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.522+0000 7f6243c00640 1 -- 172.21.15.138:0/4101479849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f623c0cd8d0 msgr2=0x7f623c151b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:15.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.522+0000 7f6243c00640 1 --2- 172.21.15.138:0/4101479849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f623c0cd8d0 0x7f623c151b90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6234033000 tx=0x7f6234004a40 comp rx=0 tx=0).stop 2024-01-21T22:25:15.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.522+0000 7f6243c00640 1 -- 172.21.15.138:0/4101479849 shutdown_connections 2024-01-21T22:25:15.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.522+0000 7f6243c00640 1 --2- 172.21.15.138:0/4101479849 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f621406b380 0x7f621406d840 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:15.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.522+0000 7f6243c00640 1 --2- 172.21.15.138:0/4101479849 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f623c0cd8d0 0x7f623c151b90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:15.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.522+0000 7f6243c00640 1 --2- 172.21.15.138:0/4101479849 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f623c0cd4f0 0x7f623c151650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:15.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.522+0000 7f6243c00640 1 --2- 172.21.15.138:0/4101479849 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f623c06f750 0x7f623c0ca5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:15.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.523+0000 7f6243c00640 1 -- 172.21.15.138:0/4101479849 >> 172.21.15.138:0/4101479849 conn(0x7f623c06ad20 msgr2=0x7f623c15ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:15.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.523+0000 7f6243c00640 1 -- 172.21.15.138:0/4101479849 shutdown_connections 2024-01-21T22:25:15.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.523+0000 7f6243c00640 1 -- 172.21.15.138:0/4101479849 wait complete. 2024-01-21T22:25:15.544 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-01-21T22:25:15.216763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219240,"kb_used_data":3080,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755480,"statfs":{"total":773094113280,"available":772869611520,"internally_reserved":0,"allocated":3153920,"data_stored":1865780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.702518"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286921+0000","last_change":"2024-01-21T22:25:04.888366+0000","last_active":"2024-01-21T22:25:07.286921+0000","last_peered":"2024-01-21T22:25:07.286921+0000","last_clean":"2024-01-21T22:25:07.286921+0000","last_became_active":"2024-01-21T22:25:04.888060+0000","last_became_peered":"2024-01-21T22:25:04.888060+0000","last_unstale":"2024-01-21T22:25:07.286921+0000","last_undegraded":"2024-01-21T22:25:07.286921+0000","last_fullsized":"2024-01-21T22:25:07.286921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:32:13.512213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298177+0000","last_change":"2024-01-21T22:25:04.843704+0000","last_active":"2024-01-21T22:25:07.298177+0000","last_peered":"2024-01-21T22:25:07.298177+0000","last_clean":"2024-01-21T22:25:07.298177+0000","last_became_active":"2024-01-21T22:25:04.843471+0000","last_became_peered":"2024-01-21T22:25:04.843471+0000","last_unstale":"2024-01-21T22:25:07.298177+0000","last_undegraded":"2024-01-21T22:25:07.298177+0000","last_fullsized":"2024-01-21T22:25:07.298177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:34:24.822768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298055+0000","last_change":"2024-01-21T22:25:04.858021+0000","last_active":"2024-01-21T22:25:07.298055+0000","last_peered":"2024-01-21T22:25:07.298055+0000","last_clean":"2024-01-21T22:25:07.298055+0000","last_became_active":"2024-01-21T22:25:04.857685+0000","last_became_peered":"2024-01-21T22:25:04.857685+0000","last_unstale":"2024-01-21T22:25:07.298055+0000","last_undegraded":"2024-01-21T22:25:07.298055+0000","last_fullsized":"2024-01-21T22:25:07.298055+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:29:02.950903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.144191+0000","last_change":"2024-01-21T22:25:04.858941+0000","last_active":"2024-01-21T22:25:07.144191+0000","last_peered":"2024-01-21T22:25:07.144191+0000","last_clean":"2024-01-21T22:25:07.144191+0000","last_became_active":"2024-01-21T22:25:04.858278+0000","last_became_peered":"2024-01-21T22:25:04.858278+0000","last_unstale":"2024-01-21T22:25:07.144191+0000","last_undegraded":"2024-01-21T22:25:07.144191+0000","last_fullsized":"2024-01-21T22:25:07.144191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T10:03:50.093635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286258+0000","last_change":"2024-01-21T22:25:04.881014+0000","last_active":"2024-01-21T22:25:07.286258+0000","last_peered":"2024-01-21T22:25:07.286258+0000","last_clean":"2024-01-21T22:25:07.286258+0000","last_became_active":"2024-01-21T22:25:04.878457+0000","last_became_peered":"2024-01-21T22:25:04.878457+0000","last_unstale":"2024-01-21T22:25:07.286258+0000","last_undegraded":"2024-01-21T22:25:07.286258+0000","last_fullsized":"2024-01-21T22:25:07.286258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:34:24.822768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.143791+0000","last_change":"2024-01-21T22:25:04.859776+0000","last_active":"2024-01-21T22:25:07.143791+0000","last_peered":"2024-01-21T22:25:07.143791+0000","last_clean":"2024-01-21T22:25:07.143791+0000","last_became_active":"2024-01-21T22:25:04.858731+0000","last_became_peered":"2024-01-21T22:25:04.858731+0000","last_unstale":"2024-01-21T22:25:07.143791+0000","last_undegraded":"2024-01-21T22:25:07.143791+0000","last_fullsized":"2024-01-21T22:25:07.143791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:01:30.091588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.285525+0000","last_change":"2024-01-21T22:25:04.881392+0000","last_active":"2024-01-21T22:25:07.285525+0000","last_peered":"2024-01-21T22:25:07.285525+0000","last_clean":"2024-01-21T22:25:07.285525+0000","last_became_active":"2024-01-21T22:25:04.881090+0000","last_became_peered":"2024-01-21T22:25:04.881090+0000","last_unstale":"2024-01-21T22:25:07.285525+0000","last_undegraded":"2024-01-21T22:25:07.285525+0000","last_fullsized":"2024-01-21T22:25:07.285525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:25:50.380424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297019+0000","last_change":"2024-01-21T22:25:04.879186+0000","last_active":"2024-01-21T22:25:07.297019+0000","last_peered":"2024-01-21T22:25:07.297019+0000","last_clean":"2024-01-21T22:25:07.297019+0000","last_became_active":"2024-01-21T22:25:04.878824+0000","last_became_peered":"2024-01-21T22:25:04.878824+0000","last_unstale":"2024-01-21T22:25:07.297019+0000","last_undegraded":"2024-01-21T22:25:07.297019+0000","last_fullsized":"2024-01-21T22:25:07.297019+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:31:25.985811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304238+0000","last_change":"2024-01-21T22:25:04.886391+0000","last_active":"2024-01-21T22:25:07.304238+0000","last_peered":"2024-01-21T22:25:07.304238+0000","last_clean":"2024-01-21T22:25:07.304238+0000","last_became_active":"2024-01-21T22:25:04.884979+0000","last_became_peered":"2024-01-21T22:25:04.884979+0000","last_unstale":"2024-01-21T22:25:07.304238+0000","last_undegraded":"2024-01-21T22:25:07.304238+0000","last_fullsized":"2024-01-21T22:25:07.304238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:36:36.009632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286546+0000","last_change":"2024-01-21T22:25:04.888126+0000","last_active":"2024-01-21T22:25:07.286546+0000","last_peered":"2024-01-21T22:25:07.286546+0000","last_clean":"2024-01-21T22:25:07.286546+0000","last_became_active":"2024-01-21T22:25:04.887631+0000","last_became_peered":"2024-01-21T22:25:04.887631+0000","last_unstale":"2024-01-21T22:25:07.286546+0000","last_undegraded":"2024-01-21T22:25:07.286546+0000","last_fullsized":"2024-01-21T22:25:07.286546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:00:02.684745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286056+0000","last_change":"2024-01-21T22:25:04.885677+0000","last_active":"2024-01-21T22:25:07.286056+0000","last_peered":"2024-01-21T22:25:07.286056+0000","last_clean":"2024-01-21T22:25:07.286056+0000","last_became_active":"2024-01-21T22:25:04.884697+0000","last_became_peered":"2024-01-21T22:25:04.884697+0000","last_unstale":"2024-01-21T22:25:07.286056+0000","last_undegraded":"2024-01-21T22:25:07.286056+0000","last_fullsized":"2024-01-21T22:25:07.286056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:44:23.749631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286124+0000","last_change":"2024-01-21T22:25:04.886799+0000","last_active":"2024-01-21T22:25:07.286124+0000","last_peered":"2024-01-21T22:25:07.286124+0000","last_clean":"2024-01-21T22:25:07.286124+0000","last_became_active":"2024-01-21T22:25:04.886387+0000","last_became_peered":"2024-01-21T22:25:04.886387+0000","last_unstale":"2024-01-21T22:25:07.286124+0000","last_undegraded":"2024-01-21T22:25:07.286124+0000","last_fullsized":"2024-01-21T22:25:07.286124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:42:39.977125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304238+0000","last_change":"2024-01-21T22:25:04.888103+0000","last_active":"2024-01-21T22:25:07.304238+0000","last_peered":"2024-01-21T22:25:07.304238+0000","last_clean":"2024-01-21T22:25:07.304238+0000","last_became_active":"2024-01-21T22:25:04.887908+0000","last_became_peered":"2024-01-21T22:25:04.887908+0000","last_unstale":"2024-01-21T22:25:07.304238+0000","last_undegraded":"2024-01-21T22:25:07.304238+0000","last_fullsized":"2024-01-21T22:25:07.304238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:23:52.444277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304247+0000","last_change":"2024-01-21T22:25:04.842750+0000","last_active":"2024-01-21T22:25:07.304247+0000","last_peered":"2024-01-21T22:25:07.304247+0000","last_clean":"2024-01-21T22:25:07.304247+0000","last_became_active":"2024-01-21T22:25:04.842512+0000","last_became_peered":"2024-01-21T22:25:04.842512+0000","last_unstale":"2024-01-21T22:25:07.304247+0000","last_undegraded":"2024-01-21T22:25:07.304247+0000","last_fullsized":"2024-01-21T22:25:07.304247+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:40:05.554704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.296486+0000","last_change":"2024-01-21T22:25:04.876592+0000","last_active":"2024-01-21T22:25:07.296486+0000","last_peered":"2024-01-21T22:25:07.296486+0000","last_clean":"2024-01-21T22:25:07.296486+0000","last_became_active":"2024-01-21T22:25:04.876305+0000","last_became_peered":"2024-01-21T22:25:04.876305+0000","last_unstale":"2024-01-21T22:25:07.296486+0000","last_undegraded":"2024-01-21T22:25:07.296486+0000","last_fullsized":"2024-01-21T22:25:07.296486+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:32.177562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093349+0000","last_change":"2024-01-21T22:25:04.878038+0000","last_active":"2024-01-21T22:25:07.093349+0000","last_peered":"2024-01-21T22:25:07.093349+0000","last_clean":"2024-01-21T22:25:07.093349+0000","last_became_active":"2024-01-21T22:25:04.877716+0000","last_became_peered":"2024-01-21T22:25:04.877716+0000","last_unstale":"2024-01-21T22:25:07.093349+0000","last_undegraded":"2024-01-21T22:25:07.093349+0000","last_fullsized":"2024-01-21T22:25:07.093349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:32:13.512213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.094445+0000","last_change":"2024-01-21T22:25:04.890537+0000","last_active":"2024-01-21T22:25:07.094445+0000","last_peered":"2024-01-21T22:25:07.094445+0000","last_clean":"2024-01-21T22:25:07.094445+0000","last_became_active":"2024-01-21T22:25:04.890298+0000","last_became_peered":"2024-01-21T22:25:04.890298+0000","last_unstale":"2024-01-21T22:25:07.094445+0000","last_undegraded":"2024-01-21T22:25:07.094445+0000","last_fullsized":"2024-01-21T22:25:07.094445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:23:57.435028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.296659+0000","last_change":"2024-01-21T22:25:04.712868+0000","last_active":"2024-01-21T22:25:07.296659+0000","last_peered":"2024-01-21T22:25:07.296659+0000","last_clean":"2024-01-21T22:25:07.296659+0000","last_became_active":"2024-01-21T22:25:04.712616+0000","last_became_peered":"2024-01-21T22:25:04.712616+0000","last_unstale":"2024-01-21T22:25:07.296659+0000","last_undegraded":"2024-01-21T22:25:07.296659+0000","last_fullsized":"2024-01-21T22:25:07.296659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:50:41.306582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.094344+0000","last_change":"2024-01-21T22:25:04.879328+0000","last_active":"2024-01-21T22:25:07.094344+0000","last_peered":"2024-01-21T22:25:07.094344+0000","last_clean":"2024-01-21T22:25:07.094344+0000","last_became_active":"2024-01-21T22:25:04.879011+0000","last_became_peered":"2024-01-21T22:25:04.879011+0000","last_unstale":"2024-01-21T22:25:07.094344+0000","last_undegraded":"2024-01-21T22:25:07.094344+0000","last_fullsized":"2024-01-21T22:25:07.094344+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:31:23.569672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286286+0000","last_change":"2024-01-21T22:25:04.885620+0000","last_active":"2024-01-21T22:25:07.286286+0000","last_peered":"2024-01-21T22:25:07.286286+0000","last_clean":"2024-01-21T22:25:07.286286+0000","last_became_active":"2024-01-21T22:25:04.884364+0000","last_became_peered":"2024-01-21T22:25:04.884364+0000","last_unstale":"2024-01-21T22:25:07.286286+0000","last_undegraded":"2024-01-21T22:25:07.286286+0000","last_fullsized":"2024-01-21T22:25:07.286286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:13:04.601911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151781+0000","last_change":"2024-01-21T22:25:04.868371+0000","last_active":"2024-01-21T22:25:07.151781+0000","last_peered":"2024-01-21T22:25:07.151781+0000","last_clean":"2024-01-21T22:25:07.151781+0000","last_became_active":"2024-01-21T22:25:04.868196+0000","last_became_peered":"2024-01-21T22:25:04.868196+0000","last_unstale":"2024-01-21T22:25:07.151781+0000","last_undegraded":"2024-01-21T22:25:07.151781+0000","last_fullsized":"2024-01-21T22:25:07.151781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:05:41.950232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.303520+0000","last_change":"2024-01-21T22:25:04.879144+0000","last_active":"2024-01-21T22:25:07.303520+0000","last_peered":"2024-01-21T22:25:07.303520+0000","last_clean":"2024-01-21T22:25:07.303520+0000","last_became_active":"2024-01-21T22:25:04.878868+0000","last_became_peered":"2024-01-21T22:25:04.878868+0000","last_unstale":"2024-01-21T22:25:07.303520+0000","last_undegraded":"2024-01-21T22:25:07.303520+0000","last_fullsized":"2024-01-21T22:25:07.303520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:55:28.069411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.296623+0000","last_change":"2024-01-21T22:25:04.715246+0000","last_active":"2024-01-21T22:25:07.296623+0000","last_peered":"2024-01-21T22:25:07.296623+0000","last_clean":"2024-01-21T22:25:07.296623+0000","last_became_active":"2024-01-21T22:25:04.714969+0000","last_became_peered":"2024-01-21T22:25:04.714969+0000","last_unstale":"2024-01-21T22:25:07.296623+0000","last_undegraded":"2024-01-21T22:25:07.296623+0000","last_fullsized":"2024-01-21T22:25:07.296623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:56:24.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093802+0000","last_change":"2024-01-21T22:25:04.877902+0000","last_active":"2024-01-21T22:25:07.093802+0000","last_peered":"2024-01-21T22:25:07.093802+0000","last_clean":"2024-01-21T22:25:07.093802+0000","last_became_active":"2024-01-21T22:25:04.876963+0000","last_became_peered":"2024-01-21T22:25:04.876963+0000","last_unstale":"2024-01-21T22:25:07.093802+0000","last_undegraded":"2024-01-21T22:25:07.093802+0000","last_fullsized":"2024-01-21T22:25:07.093802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:30:48.864454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297800+0000","last_change":"2024-01-21T22:25:04.716735+0000","last_active":"2024-01-21T22:25:07.297800+0000","last_peered":"2024-01-21T22:25:07.297800+0000","last_clean":"2024-01-21T22:25:07.297800+0000","last_became_active":"2024-01-21T22:25:04.716570+0000","last_became_peered":"2024-01-21T22:25:04.716570+0000","last_unstale":"2024-01-21T22:25:07.297800+0000","last_undegraded":"2024-01-21T22:25:07.297800+0000","last_fullsized":"2024-01-21T22:25:07.297800+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:44:55.158990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.143839+0000","last_change":"2024-01-21T22:25:04.886230+0000","last_active":"2024-01-21T22:25:07.143839+0000","last_peered":"2024-01-21T22:25:07.143839+0000","last_clean":"2024-01-21T22:25:07.143839+0000","last_became_active":"2024-01-21T22:25:04.885523+0000","last_became_peered":"2024-01-21T22:25:04.885523+0000","last_unstale":"2024-01-21T22:25:07.143839+0000","last_undegraded":"2024-01-21T22:25:07.143839+0000","last_fullsized":"2024-01-21T22:25:07.143839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:06:02.934920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304425+0000","last_change":"2024-01-21T22:25:04.842692+0000","last_active":"2024-01-21T22:25:07.304425+0000","last_peered":"2024-01-21T22:25:07.304425+0000","last_clean":"2024-01-21T22:25:07.304425+0000","last_became_active":"2024-01-21T22:25:04.842429+0000","last_became_peered":"2024-01-21T22:25:04.842429+0000","last_unstale":"2024-01-21T22:25:07.304425+0000","last_undegraded":"2024-01-21T22:25:07.304425+0000","last_fullsized":"2024-01-21T22:25:07.304425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:52:24.233709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304314+0000","last_change":"2024-01-21T22:25:04.888296+0000","last_active":"2024-01-21T22:25:07.304314+0000","last_peered":"2024-01-21T22:25:07.304314+0000","last_clean":"2024-01-21T22:25:07.304314+0000","last_became_active":"2024-01-21T22:25:04.888025+0000","last_became_peered":"2024-01-21T22:25:04.888025+0000","last_unstale":"2024-01-21T22:25:07.304314+0000","last_undegraded":"2024-01-21T22:25:07.304314+0000","last_fullsized":"2024-01-21T22:25:07.304314+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:57:44.129490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297099+0000","last_change":"2024-01-21T22:25:04.711908+0000","last_active":"2024-01-21T22:25:07.297099+0000","last_peered":"2024-01-21T22:25:07.297099+0000","last_clean":"2024-01-21T22:25:07.297099+0000","last_became_active":"2024-01-21T22:25:04.711654+0000","last_became_peered":"2024-01-21T22:25:04.711654+0000","last_unstale":"2024-01-21T22:25:07.297099+0000","last_undegraded":"2024-01-21T22:25:07.297099+0000","last_fullsized":"2024-01-21T22:25:07.297099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:20:41.612265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093463+0000","last_change":"2024-01-21T22:25:04.876757+0000","last_active":"2024-01-21T22:25:07.093463+0000","last_peered":"2024-01-21T22:25:07.093463+0000","last_clean":"2024-01-21T22:25:07.093463+0000","last_became_active":"2024-01-21T22:25:04.876012+0000","last_became_peered":"2024-01-21T22:25:04.876012+0000","last_unstale":"2024-01-21T22:25:07.093463+0000","last_undegraded":"2024-01-21T22:25:07.093463+0000","last_fullsized":"2024-01-21T22:25:07.093463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:00:56.239773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.296179+0000","last_change":"2024-01-21T22:25:04.715277+0000","last_active":"2024-01-21T22:25:07.296179+0000","last_peered":"2024-01-21T22:25:07.296179+0000","last_clean":"2024-01-21T22:25:07.296179+0000","last_became_active":"2024-01-21T22:25:04.715034+0000","last_became_peered":"2024-01-21T22:25:04.715034+0000","last_unstale":"2024-01-21T22:25:07.296179+0000","last_undegraded":"2024-01-21T22:25:07.296179+0000","last_fullsized":"2024-01-21T22:25:07.296179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:55:28.069411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.143324+0000","last_change":"2024-01-21T22:25:04.844472+0000","last_active":"2024-01-21T22:25:07.143324+0000","last_peered":"2024-01-21T22:25:07.143324+0000","last_clean":"2024-01-21T22:25:07.143324+0000","last_became_active":"2024-01-21T22:25:04.843827+0000","last_became_peered":"2024-01-21T22:25:04.843827+0000","last_unstale":"2024-01-21T22:25:07.143324+0000","last_undegraded":"2024-01-21T22:25:07.143324+0000","last_fullsized":"2024-01-21T22:25:07.143324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:58:10.552987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.303755+0000","last_change":"2024-01-21T22:25:04.713764+0000","last_active":"2024-01-21T22:25:07.303755+0000","last_peered":"2024-01-21T22:25:07.303755+0000","last_clean":"2024-01-21T22:25:07.303755+0000","last_became_active":"2024-01-21T22:25:04.712814+0000","last_became_peered":"2024-01-21T22:25:04.712814+0000","last_unstale":"2024-01-21T22:25:07.303755+0000","last_undegraded":"2024-01-21T22:25:07.303755+0000","last_fullsized":"2024-01-21T22:25:07.303755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:16:41.440927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297178+0000","last_change":"2024-01-21T22:25:03.703079+0000","last_active":"2024-01-21T22:25:07.297178+0000","last_peered":"2024-01-21T22:25:07.297178+0000","last_clean":"2024-01-21T22:25:07.297178+0000","last_became_active":"2024-01-21T22:25:03.702776+0000","last_became_peered":"2024-01-21T22:25:03.702776+0000","last_unstale":"2024-01-21T22:25:07.297178+0000","last_undegraded":"2024-01-21T22:25:07.297178+0000","last_fullsized":"2024-01-21T22:25:07.297178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:34:13.302149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151769+0000","last_change":"2024-01-21T22:25:04.877158+0000","last_active":"2024-01-21T22:25:07.151769+0000","last_peered":"2024-01-21T22:25:07.151769+0000","last_clean":"2024-01-21T22:25:07.151769+0000","last_became_active":"2024-01-21T22:25:04.876920+0000","last_became_peered":"2024-01-21T22:25:04.876920+0000","last_unstale":"2024-01-21T22:25:07.151769+0000","last_undegraded":"2024-01-21T22:25:07.151769+0000","last_fullsized":"2024-01-21T22:25:07.151769+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:46:55.451173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093458+0000","last_change":"2024-01-21T22:25:03.720178+0000","last_active":"2024-01-21T22:25:07.093458+0000","last_peered":"2024-01-21T22:25:07.093458+0000","last_clean":"2024-01-21T22:25:07.093458+0000","last_became_active":"2024-01-21T22:25:03.719909+0000","last_became_peered":"2024-01-21T22:25:03.719909+0000","last_unstale":"2024-01-21T22:25:07.093458+0000","last_undegraded":"2024-01-21T22:25:07.093458+0000","last_fullsized":"2024-01-21T22:25:07.093458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:44:53.910872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.877462+0000","last_change":"2024-01-21T22:25:03.723005+0000","last_active":"2024-01-21T22:25:05.877462+0000","last_peered":"2024-01-21T22:25:05.877462+0000","last_clean":"2024-01-21T22:25:05.877462+0000","last_became_active":"2024-01-21T22:25:03.722758+0000","last_became_peered":"2024-01-21T22:25:03.722758+0000","last_unstale":"2024-01-21T22:25:05.877462+0000","last_undegraded":"2024-01-21T22:25:05.877462+0000","last_fullsized":"2024-01-21T22:25:05.877462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:16:40.192808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297215+0000","last_change":"2024-01-21T22:25:04.844030+0000","last_active":"2024-01-21T22:25:07.297215+0000","last_peered":"2024-01-21T22:25:07.297215+0000","last_clean":"2024-01-21T22:25:07.297215+0000","last_became_active":"2024-01-21T22:25:04.843714+0000","last_became_peered":"2024-01-21T22:25:04.843714+0000","last_unstale":"2024-01-21T22:25:07.297215+0000","last_undegraded":"2024-01-21T22:25:07.297215+0000","last_fullsized":"2024-01-21T22:25:07.297215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:25:50.380424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.521757+0000","last_change":"2024-01-21T22:25:03.702767+0000","last_active":"2024-01-21T22:25:05.521757+0000","last_peered":"2024-01-21T22:25:05.521757+0000","last_clean":"2024-01-21T22:25:05.521757+0000","last_became_active":"2024-01-21T22:25:03.702140+0000","last_became_peered":"2024-01-21T22:25:03.702140+0000","last_unstale":"2024-01-21T22:25:05.521757+0000","last_undegraded":"2024-01-21T22:25:05.521757+0000","last_fullsized":"2024-01-21T22:25:05.521757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:46:54.203055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151688+0000","last_change":"2024-01-21T22:25:04.867173+0000","last_active":"2024-01-21T22:25:07.151688+0000","last_peered":"2024-01-21T22:25:07.151688+0000","last_clean":"2024-01-21T22:25:07.151688+0000","last_became_active":"2024-01-21T22:25:04.866945+0000","last_became_peered":"2024-01-21T22:25:04.866945+0000","last_unstale":"2024-01-21T22:25:07.151688+0000","last_undegraded":"2024-01-21T22:25:07.151688+0000","last_fullsized":"2024-01-21T22:25:07.151688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:15:41.725132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.882129+0000","last_change":"2024-01-21T22:25:03.715128+0000","last_active":"2024-01-21T22:25:05.882129+0000","last_peered":"2024-01-21T22:25:05.882129+0000","last_clean":"2024-01-21T22:25:05.882129+0000","last_became_active":"2024-01-21T22:25:03.714039+0000","last_became_peered":"2024-01-21T22:25:03.714039+0000","last_unstale":"2024-01-21T22:25:05.882129+0000","last_undegraded":"2024-01-21T22:25:05.882129+0000","last_fullsized":"2024-01-21T22:25:05.882129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:09:09.988399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.144063+0000","last_change":"2024-01-21T22:25:04.886534+0000","last_active":"2024-01-21T22:25:07.144063+0000","last_peered":"2024-01-21T22:25:07.144063+0000","last_clean":"2024-01-21T22:25:07.144063+0000","last_became_active":"2024-01-21T22:25:04.885963+0000","last_became_peered":"2024-01-21T22:25:04.885963+0000","last_unstale":"2024-01-21T22:25:07.144063+0000","last_undegraded":"2024-01-21T22:25:07.144063+0000","last_fullsized":"2024-01-21T22:25:07.144063+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:44:31.867220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.610812+0000","last_change":"2024-01-21T22:25:03.718237+0000","last_active":"2024-01-21T22:25:05.610812+0000","last_peered":"2024-01-21T22:25:05.610812+0000","last_clean":"2024-01-21T22:25:05.610812+0000","last_became_active":"2024-01-21T22:25:03.717937+0000","last_became_peered":"2024-01-21T22:25:03.717937+0000","last_unstale":"2024-01-21T22:25:05.610812+0000","last_undegraded":"2024-01-21T22:25:05.610812+0000","last_fullsized":"2024-01-21T22:25:05.610812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:31:24.737692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298047+0000","last_change":"2024-01-21T22:25:04.882743+0000","last_active":"2024-01-21T22:25:07.298047+0000","last_peered":"2024-01-21T22:25:07.298047+0000","last_clean":"2024-01-21T22:25:07.298047+0000","last_became_active":"2024-01-21T22:25:04.882478+0000","last_became_peered":"2024-01-21T22:25:04.882478+0000","last_unstale":"2024-01-21T22:25:07.298047+0000","last_undegraded":"2024-01-21T22:25:07.298047+0000","last_fullsized":"2024-01-21T22:25:07.298047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:42:39.977125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.146198+0000","last_change":"2024-01-21T22:25:03.703672+0000","last_active":"2024-01-21T22:25:07.146198+0000","last_peered":"2024-01-21T22:25:07.146198+0000","last_clean":"2024-01-21T22:25:07.146198+0000","last_became_active":"2024-01-21T22:25:03.703426+0000","last_became_peered":"2024-01-21T22:25:03.703426+0000","last_unstale":"2024-01-21T22:25:07.146198+0000","last_undegraded":"2024-01-21T22:25:07.146198+0000","last_fullsized":"2024-01-21T22:25:07.146198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:41:51.793206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093763+0000","last_change":"2024-01-21T22:25:04.887306+0000","last_active":"2024-01-21T22:25:07.093763+0000","last_peered":"2024-01-21T22:25:07.093763+0000","last_clean":"2024-01-21T22:25:07.093763+0000","last_became_active":"2024-01-21T22:25:04.886970+0000","last_became_peered":"2024-01-21T22:25:04.886970+0000","last_unstale":"2024-01-21T22:25:07.093763+0000","last_undegraded":"2024-01-21T22:25:07.093763+0000","last_fullsized":"2024-01-21T22:25:07.093763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:59:45.239426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"23'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.158991+0000","last_change":"2024-01-21T22:25:03.714102+0000","last_active":"2024-01-21T22:25:07.158991+0000","last_peered":"2024-01-21T22:25:07.158991+0000","last_clean":"2024-01-21T22:25:07.158991+0000","last_became_active":"2024-01-21T22:25:03.713717+0000","last_became_peered":"2024-01-21T22:25:03.713717+0000","last_unstale":"2024-01-21T22:25:07.158991+0000","last_undegraded":"2024-01-21T22:25:07.158991+0000","last_fullsized":"2024-01-21T22:25:07.158991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:32:15.181238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286154+0000","last_change":"2024-01-21T22:25:04.881307+0000","last_active":"2024-01-21T22:25:07.286154+0000","last_peered":"2024-01-21T22:25:07.286154+0000","last_clean":"2024-01-21T22:25:07.286154+0000","last_became_active":"2024-01-21T22:25:04.880472+0000","last_became_peered":"2024-01-21T22:25:04.880472+0000","last_unstale":"2024-01-21T22:25:07.286154+0000","last_undegraded":"2024-01-21T22:25:07.286154+0000","last_fullsized":"2024-01-21T22:25:07.286154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:29:02.950903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151339+0000","last_change":"2024-01-21T22:25:03.721825+0000","last_active":"2024-01-21T22:25:07.151339+0000","last_peered":"2024-01-21T22:25:07.151339+0000","last_clean":"2024-01-21T22:25:07.151339+0000","last_became_active":"2024-01-21T22:25:03.721502+0000","last_became_peered":"2024-01-21T22:25:03.721502+0000","last_unstale":"2024-01-21T22:25:07.151339+0000","last_undegraded":"2024-01-21T22:25:07.151339+0000","last_fullsized":"2024-01-21T22:25:07.151339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:09:09.988399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304218+0000","last_change":"2024-01-21T22:25:04.877455+0000","last_active":"2024-01-21T22:25:07.304218+0000","last_peered":"2024-01-21T22:25:07.304218+0000","last_clean":"2024-01-21T22:25:07.304218+0000","last_became_active":"2024-01-21T22:25:04.877257+0000","last_became_peered":"2024-01-21T22:25:04.877257+0000","last_unstale":"2024-01-21T22:25:07.304218+0000","last_undegraded":"2024-01-21T22:25:07.304218+0000","last_fullsized":"2024-01-21T22:25:07.304218+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:34:14.550267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.522179+0000","last_change":"2024-01-21T22:25:03.706077+0000","last_active":"2024-01-21T22:25:05.522179+0000","last_peered":"2024-01-21T22:25:05.522179+0000","last_clean":"2024-01-21T22:25:05.522179+0000","last_became_active":"2024-01-21T22:25:03.702798+0000","last_became_peered":"2024-01-21T22:25:03.702798+0000","last_unstale":"2024-01-21T22:25:05.522179+0000","last_undegraded":"2024-01-21T22:25:05.522179+0000","last_fullsized":"2024-01-21T22:25:05.522179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:20:40.364147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093643+0000","last_change":"2024-01-21T22:25:04.890636+0000","last_active":"2024-01-21T22:25:07.093643+0000","last_peered":"2024-01-21T22:25:07.093643+0000","last_clean":"2024-01-21T22:25:07.093643+0000","last_became_active":"2024-01-21T22:25:04.890271+0000","last_became_peered":"2024-01-21T22:25:04.890271+0000","last_unstale":"2024-01-21T22:25:07.093643+0000","last_undegraded":"2024-01-21T22:25:07.093643+0000","last_fullsized":"2024-01-21T22:25:07.093643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:15:46.754395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.609220+0000","last_change":"2024-01-21T22:25:03.717202+0000","last_active":"2024-01-21T22:25:05.609220+0000","last_peered":"2024-01-21T22:25:05.609220+0000","last_clean":"2024-01-21T22:25:05.609220+0000","last_became_active":"2024-01-21T22:25:03.716846+0000","last_became_peered":"2024-01-21T22:25:03.716846+0000","last_unstale":"2024-01-21T22:25:05.609220+0000","last_undegraded":"2024-01-21T22:25:05.609220+0000","last_fullsized":"2024-01-21T22:25:05.609220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:10.904923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.303888+0000","last_change":"2024-01-21T22:25:04.715053+0000","last_active":"2024-01-21T22:25:07.303888+0000","last_peered":"2024-01-21T22:25:07.303888+0000","last_clean":"2024-01-21T22:25:07.303888+0000","last_became_active":"2024-01-21T22:25:04.713566+0000","last_became_peered":"2024-01-21T22:25:04.713566+0000","last_unstale":"2024-01-21T22:25:07.303888+0000","last_undegraded":"2024-01-21T22:25:07.303888+0000","last_fullsized":"2024-01-21T22:25:07.303888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:50:41.306582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"23'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304071+0000","last_change":"2024-01-21T22:25:03.703017+0000","last_active":"2024-01-21T22:25:07.304071+0000","last_peered":"2024-01-21T22:25:07.304071+0000","last_clean":"2024-01-21T22:25:07.304071+0000","last_became_active":"2024-01-21T22:25:03.702742+0000","last_became_peered":"2024-01-21T22:25:03.702742+0000","last_unstale":"2024-01-21T22:25:07.304071+0000","last_undegraded":"2024-01-21T22:25:07.304071+0000","last_fullsized":"2024-01-21T22:25:07.304071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:42:03.386594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093975+0000","last_change":"2024-01-21T22:25:04.876507+0000","last_active":"2024-01-21T22:25:07.093975+0000","last_peered":"2024-01-21T22:25:07.093975+0000","last_clean":"2024-01-21T22:25:07.093975+0000","last_became_active":"2024-01-21T22:25:04.876169+0000","last_became_peered":"2024-01-21T22:25:04.876169+0000","last_unstale":"2024-01-21T22:25:07.093975+0000","last_undegraded":"2024-01-21T22:25:07.093975+0000","last_fullsized":"2024-01-21T22:25:07.093975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:12:11.958115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.152598+0000","last_change":"2024-01-21T22:25:03.720257+0000","last_active":"2024-01-21T22:25:07.152598+0000","last_peered":"2024-01-21T22:25:07.152598+0000","last_clean":"2024-01-21T22:25:07.152598+0000","last_became_active":"2024-01-21T22:25:03.720009+0000","last_became_peered":"2024-01-21T22:25:03.720009+0000","last_unstale":"2024-01-21T22:25:07.152598+0000","last_undegraded":"2024-01-21T22:25:07.152598+0000","last_fullsized":"2024-01-21T22:25:07.152598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:37:56.569677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.144048+0000","last_change":"2024-01-21T22:25:04.883112+0000","last_active":"2024-01-21T22:25:07.144048+0000","last_peered":"2024-01-21T22:25:07.144048+0000","last_clean":"2024-01-21T22:25:07.144048+0000","last_became_active":"2024-01-21T22:25:04.882767+0000","last_became_peered":"2024-01-21T22:25:04.882767+0000","last_unstale":"2024-01-21T22:25:07.144048+0000","last_undegraded":"2024-01-21T22:25:07.144048+0000","last_fullsized":"2024-01-21T22:25:07.144048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:18:03.439738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.303803+0000","last_change":"2024-01-21T22:25:03.703516+0000","last_active":"2024-01-21T22:25:07.303803+0000","last_peered":"2024-01-21T22:25:07.303803+0000","last_clean":"2024-01-21T22:25:07.303803+0000","last_became_active":"2024-01-21T22:25:03.703256+0000","last_became_peered":"2024-01-21T22:25:03.703256+0000","last_unstale":"2024-01-21T22:25:07.303803+0000","last_undegraded":"2024-01-21T22:25:07.303803+0000","last_fullsized":"2024-01-21T22:25:07.303803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:57:42.881371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286834+0000","last_change":"2024-01-21T22:25:04.884129+0000","last_active":"2024-01-21T22:25:07.286834+0000","last_peered":"2024-01-21T22:25:07.286834+0000","last_clean":"2024-01-21T22:25:07.286834+0000","last_became_active":"2024-01-21T22:25:04.883839+0000","last_became_peered":"2024-01-21T22:25:04.883839+0000","last_unstale":"2024-01-21T22:25:07.286834+0000","last_undegraded":"2024-01-21T22:25:07.286834+0000","last_fullsized":"2024-01-21T22:25:07.286834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:43:55.049706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298049+0000","last_change":"2024-01-21T22:25:03.708366+0000","last_active":"2024-01-21T22:25:07.298049+0000","last_peered":"2024-01-21T22:25:07.298049+0000","last_clean":"2024-01-21T22:25:07.298049+0000","last_became_active":"2024-01-21T22:25:03.708101+0000","last_became_peered":"2024-01-21T22:25:03.708101+0000","last_unstale":"2024-01-21T22:25:07.298049+0000","last_undegraded":"2024-01-21T22:25:07.298049+0000","last_fullsized":"2024-01-21T22:25:07.298049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:57:42.881371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286513+0000","last_change":"2024-01-21T22:25:04.877993+0000","last_active":"2024-01-21T22:25:07.286513+0000","last_peered":"2024-01-21T22:25:07.286513+0000","last_clean":"2024-01-21T22:25:07.286513+0000","last_became_active":"2024-01-21T22:25:04.877661+0000","last_became_peered":"2024-01-21T22:25:04.877661+0000","last_unstale":"2024-01-21T22:25:07.286513+0000","last_undegraded":"2024-01-21T22:25:07.286513+0000","last_fullsized":"2024-01-21T22:25:07.286513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:20:40.947205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.878348+0000","last_change":"2024-01-21T22:25:03.721559+0000","last_active":"2024-01-21T22:25:05.878348+0000","last_peered":"2024-01-21T22:25:05.878348+0000","last_clean":"2024-01-21T22:25:05.878348+0000","last_became_active":"2024-01-21T22:25:03.721281+0000","last_became_peered":"2024-01-21T22:25:03.721281+0000","last_unstale":"2024-01-21T22:25:05.878348+0000","last_undegraded":"2024-01-21T22:25:05.878348+0000","last_fullsized":"2024-01-21T22:25:05.878348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:55:19.679388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.094279+0000","last_change":"2024-01-21T22:25:04.883087+0000","last_active":"2024-01-21T22:25:07.094279+0000","last_peered":"2024-01-21T22:25:07.094279+0000","last_clean":"2024-01-21T22:25:07.094279+0000","last_became_active":"2024-01-21T22:25:04.880227+0000","last_became_peered":"2024-01-21T22:25:04.880227+0000","last_unstale":"2024-01-21T22:25:07.094279+0000","last_undegraded":"2024-01-21T22:25:07.094279+0000","last_fullsized":"2024-01-21T22:25:07.094279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:58:09.653538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.880999+0000","last_change":"2024-01-21T22:24:52.968324+0000","last_active":"2024-01-21T22:25:05.880999+0000","last_peered":"2024-01-21T22:25:05.880999+0000","last_clean":"2024-01-21T22:25:05.880999+0000","last_became_active":"2024-01-21T22:24:52.967018+0000","last_became_peered":"2024-01-21T22:24:52.967018+0000","last_unstale":"2024-01-21T22:25:05.880999+0000","last_undegraded":"2024-01-21T22:25:05.880999+0000","last_fullsized":"2024-01-21T22:25:05.880999+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:24:50.312540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:24:50.312540+0000","last_clean_scrub_stamp":"2024-01-21T22:24:50.312540+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:52:04.800073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.522445+0000","last_change":"2024-01-21T22:25:03.707407+0000","last_active":"2024-01-21T22:25:05.522445+0000","last_peered":"2024-01-21T22:25:05.522445+0000","last_clean":"2024-01-21T22:25:05.522445+0000","last_became_active":"2024-01-21T22:25:03.707085+0000","last_became_peered":"2024-01-21T22:25:03.707085+0000","last_unstale":"2024-01-21T22:25:05.522445+0000","last_undegraded":"2024-01-21T22:25:05.522445+0000","last_fullsized":"2024-01-21T22:25:05.522445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:44:22.501512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151755+0000","last_change":"2024-01-21T22:25:04.868564+0000","last_active":"2024-01-21T22:25:07.151755+0000","last_peered":"2024-01-21T22:25:07.151755+0000","last_clean":"2024-01-21T22:25:07.151755+0000","last_became_active":"2024-01-21T22:25:04.868247+0000","last_became_peered":"2024-01-21T22:25:04.868247+0000","last_unstale":"2024-01-21T22:25:07.151755+0000","last_undegraded":"2024-01-21T22:25:07.151755+0000","last_fullsized":"2024-01-21T22:25:07.151755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:30:50.039501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.521811+0000","last_change":"2024-01-21T22:25:03.706070+0000","last_active":"2024-01-21T22:25:05.521811+0000","last_peered":"2024-01-21T22:25:05.521811+0000","last_clean":"2024-01-21T22:25:05.521811+0000","last_became_active":"2024-01-21T22:25:03.705785+0000","last_became_peered":"2024-01-21T22:25:03.705785+0000","last_unstale":"2024-01-21T22:25:05.521811+0000","last_undegraded":"2024-01-21T22:25:05.521811+0000","last_fullsized":"2024-01-21T22:25:05.521811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:30.929444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304132+0000","last_change":"2024-01-21T22:25:04.845436+0000","last_active":"2024-01-21T22:25:07.304132+0000","last_peered":"2024-01-21T22:25:07.304132+0000","last_clean":"2024-01-21T22:25:07.304132+0000","last_became_active":"2024-01-21T22:25:04.845274+0000","last_became_peered":"2024-01-21T22:25:04.845274+0000","last_unstale":"2024-01-21T22:25:07.304132+0000","last_undegraded":"2024-01-21T22:25:07.304132+0000","last_fullsized":"2024-01-21T22:25:07.304132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:08:36.686787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.878258+0000","last_change":"2024-01-21T22:25:03.709174+0000","last_active":"2024-01-21T22:25:05.878258+0000","last_peered":"2024-01-21T22:25:05.878258+0000","last_clean":"2024-01-21T22:25:05.878258+0000","last_became_active":"2024-01-21T22:25:03.708894+0000","last_became_peered":"2024-01-21T22:25:03.708894+0000","last_unstale":"2024-01-21T22:25:05.878258+0000","last_undegraded":"2024-01-21T22:25:05.878258+0000","last_fullsized":"2024-01-21T22:25:05.878258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:19:26.591140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297083+0000","last_change":"2024-01-21T22:25:04.711947+0000","last_active":"2024-01-21T22:25:07.297083+0000","last_peered":"2024-01-21T22:25:07.297083+0000","last_clean":"2024-01-21T22:25:07.297083+0000","last_became_active":"2024-01-21T22:25:04.711694+0000","last_became_peered":"2024-01-21T22:25:04.711694+0000","last_unstale":"2024-01-21T22:25:07.297083+0000","last_undegraded":"2024-01-21T22:25:07.297083+0000","last_fullsized":"2024-01-21T22:25:07.297083+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:37:57.817795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.522504+0000","last_change":"2024-01-21T22:25:03.706355+0000","last_active":"2024-01-21T22:25:05.522504+0000","last_peered":"2024-01-21T22:25:05.522504+0000","last_clean":"2024-01-21T22:25:05.522504+0000","last_became_active":"2024-01-21T22:25:03.703876+0000","last_became_peered":"2024-01-21T22:25:03.703876+0000","last_unstale":"2024-01-21T22:25:05.522504+0000","last_undegraded":"2024-01-21T22:25:05.522504+0000","last_fullsized":"2024-01-21T22:25:05.522504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:42:38.729006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.285835+0000","last_change":"2024-01-21T22:25:04.885997+0000","last_active":"2024-01-21T22:25:07.285835+0000","last_peered":"2024-01-21T22:25:07.285835+0000","last_clean":"2024-01-21T22:25:07.285835+0000","last_became_active":"2024-01-21T22:25:04.884162+0000","last_became_peered":"2024-01-21T22:25:04.884162+0000","last_unstale":"2024-01-21T22:25:07.285835+0000","last_undegraded":"2024-01-21T22:25:07.285835+0000","last_fullsized":"2024-01-21T22:25:07.285835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:44:55.158990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.521383+0000","last_change":"2024-01-21T22:25:03.708300+0000","last_active":"2024-01-21T22:25:05.521383+0000","last_peered":"2024-01-21T22:25:05.521383+0000","last_clean":"2024-01-21T22:25:05.521383+0000","last_became_active":"2024-01-21T22:25:03.708036+0000","last_became_peered":"2024-01-21T22:25:03.708036+0000","last_unstale":"2024-01-21T22:25:05.521383+0000","last_undegraded":"2024-01-21T22:25:05.521383+0000","last_fullsized":"2024-01-21T22:25:05.521383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:23:51.196159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.143828+0000","last_change":"2024-01-21T22:25:04.844713+0000","last_active":"2024-01-21T22:25:07.143828+0000","last_peered":"2024-01-21T22:25:07.143828+0000","last_clean":"2024-01-21T22:25:07.143828+0000","last_became_active":"2024-01-21T22:25:04.844008+0000","last_became_peered":"2024-01-21T22:25:04.844008+0000","last_unstale":"2024-01-21T22:25:07.143828+0000","last_undegraded":"2024-01-21T22:25:07.143828+0000","last_fullsized":"2024-01-21T22:25:07.143828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:33:25.760064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.523113+0000","last_change":"2024-01-21T22:25:03.699093+0000","last_active":"2024-01-21T22:25:05.523113+0000","last_peered":"2024-01-21T22:25:05.523113+0000","last_clean":"2024-01-21T22:25:05.523113+0000","last_became_active":"2024-01-21T22:25:03.698443+0000","last_became_peered":"2024-01-21T22:25:03.698443+0000","last_unstale":"2024-01-21T22:25:05.523113+0000","last_undegraded":"2024-01-21T22:25:05.523113+0000","last_fullsized":"2024-01-21T22:25:05.523113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:20:39.699087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304411+0000","last_change":"2024-01-21T22:25:04.842733+0000","last_active":"2024-01-21T22:25:07.304411+0000","last_peered":"2024-01-21T22:25:07.304411+0000","last_clean":"2024-01-21T22:25:07.304411+0000","last_became_active":"2024-01-21T22:25:04.842479+0000","last_became_peered":"2024-01-21T22:25:04.842479+0000","last_unstale":"2024-01-21T22:25:07.304411+0000","last_undegraded":"2024-01-21T22:25:07.304411+0000","last_fullsized":"2024-01-21T22:25:07.304411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:51:06.931790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151980+0000","last_change":"2024-01-21T22:25:03.720306+0000","last_active":"2024-01-21T22:25:07.151980+0000","last_peered":"2024-01-21T22:25:07.151980+0000","last_clean":"2024-01-21T22:25:07.151980+0000","last_became_active":"2024-01-21T22:25:03.720067+0000","last_became_peered":"2024-01-21T22:25:03.720067+0000","last_unstale":"2024-01-21T22:25:07.151980+0000","last_undegraded":"2024-01-21T22:25:07.151980+0000","last_fullsized":"2024-01-21T22:25:07.151980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:52:22.985590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151977+0000","last_change":"2024-01-21T22:25:04.859628+0000","last_active":"2024-01-21T22:25:07.151977+0000","last_peered":"2024-01-21T22:25:07.151977+0000","last_clean":"2024-01-21T22:25:07.151977+0000","last_became_active":"2024-01-21T22:25:04.859270+0000","last_became_peered":"2024-01-21T22:25:04.859270+0000","last_unstale":"2024-01-21T22:25:07.151977+0000","last_undegraded":"2024-01-21T22:25:07.151977+0000","last_fullsized":"2024-01-21T22:25:07.151977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:56:24.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.282102+0000","last_change":"2024-01-21T22:25:03.722006+0000","last_active":"2024-01-21T22:25:07.282102+0000","last_peered":"2024-01-21T22:25:07.282102+0000","last_clean":"2024-01-21T22:25:07.282102+0000","last_became_active":"2024-01-21T22:25:03.721275+0000","last_became_peered":"2024-01-21T22:25:03.721275+0000","last_unstale":"2024-01-21T22:25:07.282102+0000","last_undegraded":"2024-01-21T22:25:07.282102+0000","last_fullsized":"2024-01-21T22:25:07.282102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:43:53.801587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151916+0000","last_change":"2024-01-21T22:25:04.858864+0000","last_active":"2024-01-21T22:25:07.151916+0000","last_peered":"2024-01-21T22:25:07.151916+0000","last_clean":"2024-01-21T22:25:07.151916+0000","last_became_active":"2024-01-21T22:25:04.858586+0000","last_became_peered":"2024-01-21T22:25:04.858586+0000","last_unstale":"2024-01-21T22:25:07.151916+0000","last_undegraded":"2024-01-21T22:25:07.151916+0000","last_fullsized":"2024-01-21T22:25:07.151916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:32.177562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.609941+0000","last_change":"2024-01-21T22:25:03.718436+0000","last_active":"2024-01-21T22:25:05.609941+0000","last_peered":"2024-01-21T22:25:05.609941+0000","last_clean":"2024-01-21T22:25:05.609941+0000","last_became_active":"2024-01-21T22:25:03.718171+0000","last_became_peered":"2024-01-21T22:25:03.718171+0000","last_unstale":"2024-01-21T22:25:05.609941+0000","last_undegraded":"2024-01-21T22:25:05.609941+0000","last_fullsized":"2024-01-21T22:25:05.609941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:05:40.702113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093711+0000","last_change":"2024-01-21T22:25:04.875736+0000","last_active":"2024-01-21T22:25:07.093711+0000","last_peered":"2024-01-21T22:25:07.093711+0000","last_clean":"2024-01-21T22:25:07.093711+0000","last_became_active":"2024-01-21T22:25:04.875368+0000","last_became_peered":"2024-01-21T22:25:04.875368+0000","last_unstale":"2024-01-21T22:25:07.093711+0000","last_undegraded":"2024-01-21T22:25:07.093711+0000","last_fullsized":"2024-01-21T22:25:07.093711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:39:31.273250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.877906+0000","last_change":"2024-01-21T22:25:03.722820+0000","last_active":"2024-01-21T22:25:05.877906+0000","last_peered":"2024-01-21T22:25:05.877906+0000","last_clean":"2024-01-21T22:25:05.877906+0000","last_became_active":"2024-01-21T22:25:03.722525+0000","last_became_peered":"2024-01-21T22:25:03.722525+0000","last_unstale":"2024-01-21T22:25:05.877906+0000","last_undegraded":"2024-01-21T22:25:05.877906+0000","last_fullsized":"2024-01-21T22:25:05.877906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:40:04.306585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151604+0000","last_change":"2024-01-21T22:25:04.857451+0000","last_active":"2024-01-21T22:25:07.151604+0000","last_peered":"2024-01-21T22:25:07.151604+0000","last_clean":"2024-01-21T22:25:07.151604+0000","last_became_active":"2024-01-21T22:25:04.857205+0000","last_became_peered":"2024-01-21T22:25:04.857205+0000","last_unstale":"2024-01-21T22:25:07.151604+0000","last_undegraded":"2024-01-21T22:25:07.151604+0000","last_fullsized":"2024-01-21T22:25:07.151604+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:12.153042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.877746+0000","last_change":"2024-01-21T22:25:03.708574+0000","last_active":"2024-01-21T22:25:05.877746+0000","last_peered":"2024-01-21T22:25:05.877746+0000","last_clean":"2024-01-21T22:25:05.877746+0000","last_became_active":"2024-01-21T22:25:03.708318+0000","last_became_peered":"2024-01-21T22:25:03.708318+0000","last_unstale":"2024-01-21T22:25:05.877746+0000","last_undegraded":"2024-01-21T22:25:05.877746+0000","last_fullsized":"2024-01-21T22:25:05.877746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:50:40.058463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.296286+0000","last_change":"2024-01-21T22:25:04.843079+0000","last_active":"2024-01-21T22:25:07.296286+0000","last_peered":"2024-01-21T22:25:07.296286+0000","last_clean":"2024-01-21T22:25:07.296286+0000","last_became_active":"2024-01-21T22:25:04.842815+0000","last_became_peered":"2024-01-21T22:25:04.842815+0000","last_unstale":"2024-01-21T22:25:07.296286+0000","last_undegraded":"2024-01-21T22:25:07.296286+0000","last_fullsized":"2024-01-21T22:25:07.296286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:16:41.440927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.281616+0000","last_change":"2024-01-21T22:25:03.710506+0000","last_active":"2024-01-21T22:25:07.281616+0000","last_peered":"2024-01-21T22:25:07.281616+0000","last_clean":"2024-01-21T22:25:07.281616+0000","last_became_active":"2024-01-21T22:25:03.710262+0000","last_became_peered":"2024-01-21T22:25:03.710262+0000","last_unstale":"2024-01-21T22:25:07.281616+0000","last_undegraded":"2024-01-21T22:25:07.281616+0000","last_fullsized":"2024-01-21T22:25:07.281616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:25:49.132306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093352+0000","last_change":"2024-01-21T22:25:04.876469+0000","last_active":"2024-01-21T22:25:07.093352+0000","last_peered":"2024-01-21T22:25:07.093352+0000","last_clean":"2024-01-21T22:25:07.093352+0000","last_became_active":"2024-01-21T22:25:04.875619+0000","last_became_peered":"2024-01-21T22:25:04.875619+0000","last_unstale":"2024-01-21T22:25:07.093352+0000","last_undegraded":"2024-01-21T22:25:07.093352+0000","last_fullsized":"2024-01-21T22:25:07.093352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:30:20.167607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.094159+0000","last_change":"2024-01-21T22:25:04.891067+0000","last_active":"2024-01-21T22:25:07.094159+0000","last_peered":"2024-01-21T22:25:07.094159+0000","last_clean":"2024-01-21T22:25:07.094159+0000","last_became_active":"2024-01-21T22:25:04.890515+0000","last_became_peered":"2024-01-21T22:25:04.890515+0000","last_unstale":"2024-01-21T22:25:07.094159+0000","last_undegraded":"2024-01-21T22:25:07.094159+0000","last_fullsized":"2024-01-21T22:25:07.094159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:25:48.726591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.158438+0000","last_change":"2024-01-21T22:25:03.719412+0000","last_active":"2024-01-21T22:25:07.158438+0000","last_peered":"2024-01-21T22:25:07.158438+0000","last_clean":"2024-01-21T22:25:07.158438+0000","last_became_active":"2024-01-21T22:25:03.718163+0000","last_became_peered":"2024-01-21T22:25:03.718163+0000","last_unstale":"2024-01-21T22:25:07.158438+0000","last_undegraded":"2024-01-21T22:25:07.158438+0000","last_fullsized":"2024-01-21T22:25:07.158438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:56:22.817420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304192+0000","last_change":"2024-01-21T22:25:04.889166+0000","last_active":"2024-01-21T22:25:07.304192+0000","last_peered":"2024-01-21T22:25:07.304192+0000","last_clean":"2024-01-21T22:25:07.304192+0000","last_became_active":"2024-01-21T22:25:04.888910+0000","last_became_peered":"2024-01-21T22:25:04.888910+0000","last_unstale":"2024-01-21T22:25:07.304192+0000","last_undegraded":"2024-01-21T22:25:07.304192+0000","last_fullsized":"2024-01-21T22:25:07.304192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:55:53.892439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.156719+0000","last_change":"2024-01-21T22:25:03.718932+0000","last_active":"2024-01-21T22:25:07.156719+0000","last_peered":"2024-01-21T22:25:07.156719+0000","last_clean":"2024-01-21T22:25:07.156719+0000","last_became_active":"2024-01-21T22:25:03.718649+0000","last_became_peered":"2024-01-21T22:25:03.718649+0000","last_unstale":"2024-01-21T22:25:07.156719+0000","last_undegraded":"2024-01-21T22:25:07.156719+0000","last_fullsized":"2024-01-21T22:25:07.156719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:30.929444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297922+0000","last_change":"2024-01-21T22:25:04.715429+0000","last_active":"2024-01-21T22:25:07.297922+0000","last_peered":"2024-01-21T22:25:07.297922+0000","last_clean":"2024-01-21T22:25:07.297922+0000","last_became_active":"2024-01-21T22:25:04.714720+0000","last_became_peered":"2024-01-21T22:25:04.714720+0000","last_unstale":"2024-01-21T22:25:07.297922+0000","last_undegraded":"2024-01-21T22:25:07.297922+0000","last_fullsized":"2024-01-21T22:25:07.297922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:44:23.749631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298298+0000","last_change":"2024-01-21T22:25:03.703289+0000","last_active":"2024-01-21T22:25:07.298298+0000","last_peered":"2024-01-21T22:25:07.298298+0000","last_clean":"2024-01-21T22:25:07.298298+0000","last_became_active":"2024-01-21T22:25:03.703020+0000","last_became_peered":"2024-01-21T22:25:03.703020+0000","last_unstale":"2024-01-21T22:25:07.298298+0000","last_undegraded":"2024-01-21T22:25:07.298298+0000","last_fullsized":"2024-01-21T22:25:07.298298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:10.904923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298304+0000","last_change":"2024-01-21T22:25:04.715894+0000","last_active":"2024-01-21T22:25:07.298304+0000","last_peered":"2024-01-21T22:25:07.298304+0000","last_clean":"2024-01-21T22:25:07.298304+0000","last_became_active":"2024-01-21T22:25:04.715652+0000","last_became_peered":"2024-01-21T22:25:04.715652+0000","last_unstale":"2024-01-21T22:25:07.298304+0000","last_undegraded":"2024-01-21T22:25:07.298304+0000","last_fullsized":"2024-01-21T22:25:07.298304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:13:04.601911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"23'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.303733+0000","last_change":"2024-01-21T22:25:03.702339+0000","last_active":"2024-01-21T22:25:07.303733+0000","last_peered":"2024-01-21T22:25:07.303733+0000","last_clean":"2024-01-21T22:25:07.303733+0000","last_became_active":"2024-01-21T22:25:03.702087+0000","last_became_peered":"2024-01-21T22:25:03.702087+0000","last_unstale":"2024-01-21T22:25:07.303733+0000","last_undegraded":"2024-01-21T22:25:07.303733+0000","last_fullsized":"2024-01-21T22:25:07.303733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:51:05.683671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":245760,"data_stored":85152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]}]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":85028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":839680,"data_stored":675488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":87312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542154,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":85610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.304}]}]},{"osd":2,"up_from":14,"seq":60129542152,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":249856,"data_stored":86640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.371}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5720000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":835584,"data_stored":675466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.389}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]}]},{"osd":0,"up_from":14,"seq":60129542154,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":245760,"data_stored":85084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-21T22:25:15.548 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-21T22:25:15.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.794+0000 7f93eb400640 1 Processor -- start 2024-01-21T22:25:15.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.794+0000 7f93eb400640 1 -- start start 2024-01-21T22:25:15.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.795+0000 7f93eb400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f93e4151a30 0x7f93e40c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:15.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.796+0000 7f93eb400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f93e4151e10 0x7f93e415ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:15.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.796+0000 7f93eb400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e41583d0 con 0x7f93e415b3a0 2024-01-21T22:25:15.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.796+0000 7f93eb400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e406c7b0 con 0x7f93e4151e10 2024-01-21T22:25:15.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.796+0000 7f93eb400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e4157890 con 0x7f93e4151a30 2024-01-21T22:25:15.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.796+0000 7f93e3e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f93e4151e10 0x7f93e415ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:15.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.796+0000 7f93e8a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f93e4151a30 0x7f93e40c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:15.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.796+0000 7f93e9400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f93e415b3a0 0x7f93e40c47c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:60674/0 (socket says 172.21.15.138:60674) 2024-01-21T22:25:15.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.796+0000 7f93e9400640 1 -- 172.21.15.138:0/908247094 learned_addr learned my addr 172.21.15.138:0/908247094 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:15.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.799+0000 7f93e3e00640 1 -- 172.21.15.138:0/908247094 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f93e4151a30 msgr2=0x7f93e40c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:15.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.799+0000 7f93e3e00640 1 --2- 172.21.15.138:0/908247094 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f93e4151a30 0x7f93e40c5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:15.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.799+0000 7f93e3e00640 1 -- 172.21.15.138:0/908247094 >> v1:172.21.15.138:6789/0 conn(0x7f93e415b3a0 legacy=0x7f93e40c47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:15.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.799+0000 7f93e3e00640 1 -- 172.21.15.138:0/908247094 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e40c5ff0 con 0x7f93e4151e10 2024-01-21T22:25:15.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.801+0000 7f93e3e00640 1 --2- 172.21.15.138:0/908247094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e4151e10 0x7f93e415ae60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f93cc0097d0 tx=0x7f93cc02b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11a207b782c97548 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:15.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.805+0000 7f93e3400640 1 -- 172.21.15.138:0/908247094 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc031890 con 0x7f93e4151e10 2024-01-21T22:25:15.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.805+0000 7f93e3400640 1 -- 172.21.15.138:0/908247094 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93cc0319f0 con 0x7f93e4151e10 2024-01-21T22:25:15.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.805+0000 7f93e3400640 1 -- 172.21.15.138:0/908247094 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc031d80 con 0x7f93e4151e10 2024-01-21T22:25:15.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.806+0000 7f93eb400640 1 -- 172.21.15.138:0/908247094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e4151e10 msgr2=0x7f93e415ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:15.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.806+0000 7f93eb400640 1 --2- 172.21.15.138:0/908247094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e4151e10 0x7f93e415ae60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f93cc0097d0 tx=0x7f93cc02b930 comp rx=0 tx=0).stop 2024-01-21T22:25:15.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.806+0000 7f93eb400640 1 -- 172.21.15.138:0/908247094 shutdown_connections 2024-01-21T22:25:15.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.807+0000 7f93eb400640 1 --2- 172.21.15.138:0/908247094 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e4151e10 0x7f93e415ae60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:15.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.807+0000 7f93eb400640 1 --2- 172.21.15.138:0/908247094 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f93e4151a30 0x7f93e40c5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:15.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.807+0000 7f93eb400640 1 -- 172.21.15.138:0/908247094 >> 172.21.15.138:0/908247094 conn(0x7f93e40cb0b0 msgr2=0x7f93e40cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:15.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.807+0000 7f93eb400640 1 -- 172.21.15.138:0/908247094 shutdown_connections 2024-01-21T22:25:15.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.807+0000 7f93eb400640 1 -- 172.21.15.138:0/908247094 wait complete. 2024-01-21T22:25:15.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.809+0000 7f93eb400640 1 Processor -- start 2024-01-21T22:25:15.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.809+0000 7f93eb400640 1 -- start start 2024-01-21T22:25:15.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.811+0000 7f93eb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93e4151a30 0x7f93e4158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:15.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.811+0000 7f93eb400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e415b3a0 0x7f93e41f2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:15.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.811+0000 7f93eb400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f93e415aa10 0x7f93e41f3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:15.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.811+0000 7f93eb400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e415b1c0 con 0x7f93e4151a30 2024-01-21T22:25:15.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.811+0000 7f93eb400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e415aee0 con 0x7f93e415b3a0 2024-01-21T22:25:15.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.811+0000 7f93eb400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e4144f70 con 0x7f93e415aa10 2024-01-21T22:25:15.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.811+0000 7f93e8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93e4151a30 0x7f93e4158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:15.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.812+0000 7f93e3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e415b3a0 0x7f93e41f2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:15.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.812+0000 7f93e9400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f93e415aa10 0x7f93e41f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:15.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.812+0000 7f93e8a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93e4151a30 0x7f93e4158a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43562/0 (socket says 172.21.15.138:43562) 2024-01-21T22:25:15.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.812+0000 7f93e8a00640 1 -- 172.21.15.138:0/3400243987 learned_addr learned my addr 172.21.15.138:0/3400243987 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:15.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.815+0000 7f93e3e00640 1 -- 172.21.15.138:0/3400243987 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f93e415aa10 msgr2=0x7f93e41f3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:15.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.815+0000 7f93e3e00640 1 --2- 172.21.15.138:0/3400243987 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f93e415aa10 0x7f93e41f3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:15.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.815+0000 7f93e3e00640 1 -- 172.21.15.138:0/3400243987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93e4151a30 msgr2=0x7f93e4158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:15.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.815+0000 7f93e3e00640 1 --2- 172.21.15.138:0/3400243987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93e4151a30 0x7f93e4158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:15.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.815+0000 7f93e3e00640 1 -- 172.21.15.138:0/3400243987 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93cc009400 con 0x7f93e415b3a0 2024-01-21T22:25:15.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.815+0000 7f93e8a00640 1 --2- 172.21.15.138:0/3400243987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93e4151a30 0x7f93e4158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-21T22:25:15.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.818+0000 7f93e3e00640 1 --2- 172.21.15.138:0/3400243987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e415b3a0 0x7f93e41f2b30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f93cc033000 tx=0x7f93cc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:15.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.820+0000 7f93e0c00640 1 -- 172.21.15.138:0/3400243987 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc03bd30 con 0x7f93e415b3a0 2024-01-21T22:25:15.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.820+0000 7f93eb400640 1 -- 172.21.15.138:0/3400243987 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e4145250 con 0x7f93e415b3a0 2024-01-21T22:25:15.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.820+0000 7f93eb400640 1 -- 172.21.15.138:0/3400243987 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e4145810 con 0x7f93e415b3a0 2024-01-21T22:25:15.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.821+0000 7f93e0c00640 1 -- 172.21.15.138:0/3400243987 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93cc03be90 con 0x7f93e415b3a0 2024-01-21T22:25:15.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.822+0000 7f93e0c00640 1 -- 172.21.15.138:0/3400243987 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc04d5f0 con 0x7f93e415b3a0 2024-01-21T22:25:15.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:15.823+0000 7f93eb400640 1 -- 172.21.15.138:0/3400243987 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e4071f10 con 0x7f93e415b3a0 2024-01-21T22:25:16.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.072+0000 7f93e0c00640 1 -- 172.21.15.138:0/3400243987 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f93cc04d750 con 0x7f93e415b3a0 2024-01-21T22:25:16.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.073+0000 7f93e0c00640 1 --2- 172.21.15.138:0/3400243987 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f93bc06b330 0x7f93bc06d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:16.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.074+0000 7f93e8a00640 1 --2- 172.21.15.138:0/3400243987 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f93bc06b330 0x7f93bc06d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:16.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.075+0000 7f93e8a00640 1 --2- 172.21.15.138:0/3400243987 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f93bc06b330 0x7f93bc06d7f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f93d40112c0 tx=0x7f93d4009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:16.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.076+0000 7f93e0c00640 1 -- 172.21.15.138:0/3400243987 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93cc07aa90 con 0x7f93e415b3a0 2024-01-21T22:25:16.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.091+0000 7f93e0c00640 1 -- 172.21.15.138:0/3400243987 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f93cc076dd0 con 0x7f93e415b3a0 2024-01-21T22:25:16.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.300+0000 7f93eb400640 1 -- 172.21.15.138:0/3400243987 --> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f93e4157aa0 con 0x7f93bc06b330 2024-01-21T22:25:16.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.308+0000 7f93e0c00640 1 -- 172.21.15.138:0/3400243987 <== mgr.4111 v2:172.21.15.152:6800/314625564 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264526 (secure 0 0 0) 0x7f93d4009d20 con 0x7f93bc06b330 2024-01-21T22:25:16.313 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:25:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-01-21T22:25:16.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.313+0000 7f93eb400640 1 -- 172.21.15.138:0/3400243987 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f93bc06b330 msgr2=0x7f93bc06d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:16.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.313+0000 7f93eb400640 1 --2- 172.21.15.138:0/3400243987 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f93bc06b330 0x7f93bc06d7f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f93d40112c0 tx=0x7f93d4009190 comp rx=0 tx=0).stop 2024-01-21T22:25:16.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.313+0000 7f93eb400640 1 -- 172.21.15.138:0/3400243987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e415b3a0 msgr2=0x7f93e41f2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:16.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.313+0000 7f93eb400640 1 --2- 172.21.15.138:0/3400243987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e415b3a0 0x7f93e41f2b30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f93cc033000 tx=0x7f93cc0046c0 comp rx=0 tx=0).stop 2024-01-21T22:25:16.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.313+0000 7f93eb400640 1 -- 172.21.15.138:0/3400243987 shutdown_connections 2024-01-21T22:25:16.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.313+0000 7f93eb400640 1 --2- 172.21.15.138:0/3400243987 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f93bc06b330 0x7f93bc06d7f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:16.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.313+0000 7f93eb400640 1 --2- 172.21.15.138:0/3400243987 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f93e415aa10 0x7f93e41f3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:16.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.313+0000 7f93eb400640 1 --2- 172.21.15.138:0/3400243987 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f93e415b3a0 0x7f93e41f2b30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:16.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.313+0000 7f93eb400640 1 --2- 172.21.15.138:0/3400243987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f93e4151a30 0x7f93e4158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:16.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.313+0000 7f93eb400640 1 -- 172.21.15.138:0/3400243987 >> 172.21.15.138:0/3400243987 conn(0x7f93e40cb0b0 msgr2=0x7f93e40c5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:16.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.313+0000 7f93eb400640 1 -- 172.21.15.138:0/3400243987 shutdown_connections 2024-01-21T22:25:16.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.314+0000 7f93eb400640 1 -- 172.21.15.138:0/3400243987 wait complete. 2024-01-21T22:25:16.334 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-01-21T22:25:15.216763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219240,"kb_used_data":3080,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755480,"statfs":{"total":773094113280,"available":772869611520,"internally_reserved":0,"allocated":3153920,"data_stored":1865780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.702518"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286921+0000","last_change":"2024-01-21T22:25:04.888366+0000","last_active":"2024-01-21T22:25:07.286921+0000","last_peered":"2024-01-21T22:25:07.286921+0000","last_clean":"2024-01-21T22:25:07.286921+0000","last_became_active":"2024-01-21T22:25:04.888060+0000","last_became_peered":"2024-01-21T22:25:04.888060+0000","last_unstale":"2024-01-21T22:25:07.286921+0000","last_undegraded":"2024-01-21T22:25:07.286921+0000","last_fullsized":"2024-01-21T22:25:07.286921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:32:13.512213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298177+0000","last_change":"2024-01-21T22:25:04.843704+0000","last_active":"2024-01-21T22:25:07.298177+0000","last_peered":"2024-01-21T22:25:07.298177+0000","last_clean":"2024-01-21T22:25:07.298177+0000","last_became_active":"2024-01-21T22:25:04.843471+0000","last_became_peered":"2024-01-21T22:25:04.843471+0000","last_unstale":"2024-01-21T22:25:07.298177+0000","last_undegraded":"2024-01-21T22:25:07.298177+0000","last_fullsized":"2024-01-21T22:25:07.298177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:34:24.822768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298055+0000","last_change":"2024-01-21T22:25:04.858021+0000","last_active":"2024-01-21T22:25:07.298055+0000","last_peered":"2024-01-21T22:25:07.298055+0000","last_clean":"2024-01-21T22:25:07.298055+0000","last_became_active":"2024-01-21T22:25:04.857685+0000","last_became_peered":"2024-01-21T22:25:04.857685+0000","last_unstale":"2024-01-21T22:25:07.298055+0000","last_undegraded":"2024-01-21T22:25:07.298055+0000","last_fullsized":"2024-01-21T22:25:07.298055+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:29:02.950903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.144191+0000","last_change":"2024-01-21T22:25:04.858941+0000","last_active":"2024-01-21T22:25:07.144191+0000","last_peered":"2024-01-21T22:25:07.144191+0000","last_clean":"2024-01-21T22:25:07.144191+0000","last_became_active":"2024-01-21T22:25:04.858278+0000","last_became_peered":"2024-01-21T22:25:04.858278+0000","last_unstale":"2024-01-21T22:25:07.144191+0000","last_undegraded":"2024-01-21T22:25:07.144191+0000","last_fullsized":"2024-01-21T22:25:07.144191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T10:03:50.093635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286258+0000","last_change":"2024-01-21T22:25:04.881014+0000","last_active":"2024-01-21T22:25:07.286258+0000","last_peered":"2024-01-21T22:25:07.286258+0000","last_clean":"2024-01-21T22:25:07.286258+0000","last_became_active":"2024-01-21T22:25:04.878457+0000","last_became_peered":"2024-01-21T22:25:04.878457+0000","last_unstale":"2024-01-21T22:25:07.286258+0000","last_undegraded":"2024-01-21T22:25:07.286258+0000","last_fullsized":"2024-01-21T22:25:07.286258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:34:24.822768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.143791+0000","last_change":"2024-01-21T22:25:04.859776+0000","last_active":"2024-01-21T22:25:07.143791+0000","last_peered":"2024-01-21T22:25:07.143791+0000","last_clean":"2024-01-21T22:25:07.143791+0000","last_became_active":"2024-01-21T22:25:04.858731+0000","last_became_peered":"2024-01-21T22:25:04.858731+0000","last_unstale":"2024-01-21T22:25:07.143791+0000","last_undegraded":"2024-01-21T22:25:07.143791+0000","last_fullsized":"2024-01-21T22:25:07.143791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:01:30.091588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.285525+0000","last_change":"2024-01-21T22:25:04.881392+0000","last_active":"2024-01-21T22:25:07.285525+0000","last_peered":"2024-01-21T22:25:07.285525+0000","last_clean":"2024-01-21T22:25:07.285525+0000","last_became_active":"2024-01-21T22:25:04.881090+0000","last_became_peered":"2024-01-21T22:25:04.881090+0000","last_unstale":"2024-01-21T22:25:07.285525+0000","last_undegraded":"2024-01-21T22:25:07.285525+0000","last_fullsized":"2024-01-21T22:25:07.285525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:25:50.380424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297019+0000","last_change":"2024-01-21T22:25:04.879186+0000","last_active":"2024-01-21T22:25:07.297019+0000","last_peered":"2024-01-21T22:25:07.297019+0000","last_clean":"2024-01-21T22:25:07.297019+0000","last_became_active":"2024-01-21T22:25:04.878824+0000","last_became_peered":"2024-01-21T22:25:04.878824+0000","last_unstale":"2024-01-21T22:25:07.297019+0000","last_undegraded":"2024-01-21T22:25:07.297019+0000","last_fullsized":"2024-01-21T22:25:07.297019+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:31:25.985811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304238+0000","last_change":"2024-01-21T22:25:04.886391+0000","last_active":"2024-01-21T22:25:07.304238+0000","last_peered":"2024-01-21T22:25:07.304238+0000","last_clean":"2024-01-21T22:25:07.304238+0000","last_became_active":"2024-01-21T22:25:04.884979+0000","last_became_peered":"2024-01-21T22:25:04.884979+0000","last_unstale":"2024-01-21T22:25:07.304238+0000","last_undegraded":"2024-01-21T22:25:07.304238+0000","last_fullsized":"2024-01-21T22:25:07.304238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:36:36.009632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286546+0000","last_change":"2024-01-21T22:25:04.888126+0000","last_active":"2024-01-21T22:25:07.286546+0000","last_peered":"2024-01-21T22:25:07.286546+0000","last_clean":"2024-01-21T22:25:07.286546+0000","last_became_active":"2024-01-21T22:25:04.887631+0000","last_became_peered":"2024-01-21T22:25:04.887631+0000","last_unstale":"2024-01-21T22:25:07.286546+0000","last_undegraded":"2024-01-21T22:25:07.286546+0000","last_fullsized":"2024-01-21T22:25:07.286546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:00:02.684745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286056+0000","last_change":"2024-01-21T22:25:04.885677+0000","last_active":"2024-01-21T22:25:07.286056+0000","last_peered":"2024-01-21T22:25:07.286056+0000","last_clean":"2024-01-21T22:25:07.286056+0000","last_became_active":"2024-01-21T22:25:04.884697+0000","last_became_peered":"2024-01-21T22:25:04.884697+0000","last_unstale":"2024-01-21T22:25:07.286056+0000","last_undegraded":"2024-01-21T22:25:07.286056+0000","last_fullsized":"2024-01-21T22:25:07.286056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:44:23.749631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286124+0000","last_change":"2024-01-21T22:25:04.886799+0000","last_active":"2024-01-21T22:25:07.286124+0000","last_peered":"2024-01-21T22:25:07.286124+0000","last_clean":"2024-01-21T22:25:07.286124+0000","last_became_active":"2024-01-21T22:25:04.886387+0000","last_became_peered":"2024-01-21T22:25:04.886387+0000","last_unstale":"2024-01-21T22:25:07.286124+0000","last_undegraded":"2024-01-21T22:25:07.286124+0000","last_fullsized":"2024-01-21T22:25:07.286124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:42:39.977125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304238+0000","last_change":"2024-01-21T22:25:04.888103+0000","last_active":"2024-01-21T22:25:07.304238+0000","last_peered":"2024-01-21T22:25:07.304238+0000","last_clean":"2024-01-21T22:25:07.304238+0000","last_became_active":"2024-01-21T22:25:04.887908+0000","last_became_peered":"2024-01-21T22:25:04.887908+0000","last_unstale":"2024-01-21T22:25:07.304238+0000","last_undegraded":"2024-01-21T22:25:07.304238+0000","last_fullsized":"2024-01-21T22:25:07.304238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:23:52.444277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304247+0000","last_change":"2024-01-21T22:25:04.842750+0000","last_active":"2024-01-21T22:25:07.304247+0000","last_peered":"2024-01-21T22:25:07.304247+0000","last_clean":"2024-01-21T22:25:07.304247+0000","last_became_active":"2024-01-21T22:25:04.842512+0000","last_became_peered":"2024-01-21T22:25:04.842512+0000","last_unstale":"2024-01-21T22:25:07.304247+0000","last_undegraded":"2024-01-21T22:25:07.304247+0000","last_fullsized":"2024-01-21T22:25:07.304247+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:40:05.554704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.296486+0000","last_change":"2024-01-21T22:25:04.876592+0000","last_active":"2024-01-21T22:25:07.296486+0000","last_peered":"2024-01-21T22:25:07.296486+0000","last_clean":"2024-01-21T22:25:07.296486+0000","last_became_active":"2024-01-21T22:25:04.876305+0000","last_became_peered":"2024-01-21T22:25:04.876305+0000","last_unstale":"2024-01-21T22:25:07.296486+0000","last_undegraded":"2024-01-21T22:25:07.296486+0000","last_fullsized":"2024-01-21T22:25:07.296486+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:32.177562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093349+0000","last_change":"2024-01-21T22:25:04.878038+0000","last_active":"2024-01-21T22:25:07.093349+0000","last_peered":"2024-01-21T22:25:07.093349+0000","last_clean":"2024-01-21T22:25:07.093349+0000","last_became_active":"2024-01-21T22:25:04.877716+0000","last_became_peered":"2024-01-21T22:25:04.877716+0000","last_unstale":"2024-01-21T22:25:07.093349+0000","last_undegraded":"2024-01-21T22:25:07.093349+0000","last_fullsized":"2024-01-21T22:25:07.093349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:32:13.512213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.094445+0000","last_change":"2024-01-21T22:25:04.890537+0000","last_active":"2024-01-21T22:25:07.094445+0000","last_peered":"2024-01-21T22:25:07.094445+0000","last_clean":"2024-01-21T22:25:07.094445+0000","last_became_active":"2024-01-21T22:25:04.890298+0000","last_became_peered":"2024-01-21T22:25:04.890298+0000","last_unstale":"2024-01-21T22:25:07.094445+0000","last_undegraded":"2024-01-21T22:25:07.094445+0000","last_fullsized":"2024-01-21T22:25:07.094445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:23:57.435028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.296659+0000","last_change":"2024-01-21T22:25:04.712868+0000","last_active":"2024-01-21T22:25:07.296659+0000","last_peered":"2024-01-21T22:25:07.296659+0000","last_clean":"2024-01-21T22:25:07.296659+0000","last_became_active":"2024-01-21T22:25:04.712616+0000","last_became_peered":"2024-01-21T22:25:04.712616+0000","last_unstale":"2024-01-21T22:25:07.296659+0000","last_undegraded":"2024-01-21T22:25:07.296659+0000","last_fullsized":"2024-01-21T22:25:07.296659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:50:41.306582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.094344+0000","last_change":"2024-01-21T22:25:04.879328+0000","last_active":"2024-01-21T22:25:07.094344+0000","last_peered":"2024-01-21T22:25:07.094344+0000","last_clean":"2024-01-21T22:25:07.094344+0000","last_became_active":"2024-01-21T22:25:04.879011+0000","last_became_peered":"2024-01-21T22:25:04.879011+0000","last_unstale":"2024-01-21T22:25:07.094344+0000","last_undegraded":"2024-01-21T22:25:07.094344+0000","last_fullsized":"2024-01-21T22:25:07.094344+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:31:23.569672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286286+0000","last_change":"2024-01-21T22:25:04.885620+0000","last_active":"2024-01-21T22:25:07.286286+0000","last_peered":"2024-01-21T22:25:07.286286+0000","last_clean":"2024-01-21T22:25:07.286286+0000","last_became_active":"2024-01-21T22:25:04.884364+0000","last_became_peered":"2024-01-21T22:25:04.884364+0000","last_unstale":"2024-01-21T22:25:07.286286+0000","last_undegraded":"2024-01-21T22:25:07.286286+0000","last_fullsized":"2024-01-21T22:25:07.286286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:13:04.601911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151781+0000","last_change":"2024-01-21T22:25:04.868371+0000","last_active":"2024-01-21T22:25:07.151781+0000","last_peered":"2024-01-21T22:25:07.151781+0000","last_clean":"2024-01-21T22:25:07.151781+0000","last_became_active":"2024-01-21T22:25:04.868196+0000","last_became_peered":"2024-01-21T22:25:04.868196+0000","last_unstale":"2024-01-21T22:25:07.151781+0000","last_undegraded":"2024-01-21T22:25:07.151781+0000","last_fullsized":"2024-01-21T22:25:07.151781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:05:41.950232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.303520+0000","last_change":"2024-01-21T22:25:04.879144+0000","last_active":"2024-01-21T22:25:07.303520+0000","last_peered":"2024-01-21T22:25:07.303520+0000","last_clean":"2024-01-21T22:25:07.303520+0000","last_became_active":"2024-01-21T22:25:04.878868+0000","last_became_peered":"2024-01-21T22:25:04.878868+0000","last_unstale":"2024-01-21T22:25:07.303520+0000","last_undegraded":"2024-01-21T22:25:07.303520+0000","last_fullsized":"2024-01-21T22:25:07.303520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:55:28.069411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.296623+0000","last_change":"2024-01-21T22:25:04.715246+0000","last_active":"2024-01-21T22:25:07.296623+0000","last_peered":"2024-01-21T22:25:07.296623+0000","last_clean":"2024-01-21T22:25:07.296623+0000","last_became_active":"2024-01-21T22:25:04.714969+0000","last_became_peered":"2024-01-21T22:25:04.714969+0000","last_unstale":"2024-01-21T22:25:07.296623+0000","last_undegraded":"2024-01-21T22:25:07.296623+0000","last_fullsized":"2024-01-21T22:25:07.296623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:56:24.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093802+0000","last_change":"2024-01-21T22:25:04.877902+0000","last_active":"2024-01-21T22:25:07.093802+0000","last_peered":"2024-01-21T22:25:07.093802+0000","last_clean":"2024-01-21T22:25:07.093802+0000","last_became_active":"2024-01-21T22:25:04.876963+0000","last_became_peered":"2024-01-21T22:25:04.876963+0000","last_unstale":"2024-01-21T22:25:07.093802+0000","last_undegraded":"2024-01-21T22:25:07.093802+0000","last_fullsized":"2024-01-21T22:25:07.093802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:30:48.864454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297800+0000","last_change":"2024-01-21T22:25:04.716735+0000","last_active":"2024-01-21T22:25:07.297800+0000","last_peered":"2024-01-21T22:25:07.297800+0000","last_clean":"2024-01-21T22:25:07.297800+0000","last_became_active":"2024-01-21T22:25:04.716570+0000","last_became_peered":"2024-01-21T22:25:04.716570+0000","last_unstale":"2024-01-21T22:25:07.297800+0000","last_undegraded":"2024-01-21T22:25:07.297800+0000","last_fullsized":"2024-01-21T22:25:07.297800+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:44:55.158990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.143839+0000","last_change":"2024-01-21T22:25:04.886230+0000","last_active":"2024-01-21T22:25:07.143839+0000","last_peered":"2024-01-21T22:25:07.143839+0000","last_clean":"2024-01-21T22:25:07.143839+0000","last_became_active":"2024-01-21T22:25:04.885523+0000","last_became_peered":"2024-01-21T22:25:04.885523+0000","last_unstale":"2024-01-21T22:25:07.143839+0000","last_undegraded":"2024-01-21T22:25:07.143839+0000","last_fullsized":"2024-01-21T22:25:07.143839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:06:02.934920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304425+0000","last_change":"2024-01-21T22:25:04.842692+0000","last_active":"2024-01-21T22:25:07.304425+0000","last_peered":"2024-01-21T22:25:07.304425+0000","last_clean":"2024-01-21T22:25:07.304425+0000","last_became_active":"2024-01-21T22:25:04.842429+0000","last_became_peered":"2024-01-21T22:25:04.842429+0000","last_unstale":"2024-01-21T22:25:07.304425+0000","last_undegraded":"2024-01-21T22:25:07.304425+0000","last_fullsized":"2024-01-21T22:25:07.304425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:52:24.233709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304314+0000","last_change":"2024-01-21T22:25:04.888296+0000","last_active":"2024-01-21T22:25:07.304314+0000","last_peered":"2024-01-21T22:25:07.304314+0000","last_clean":"2024-01-21T22:25:07.304314+0000","last_became_active":"2024-01-21T22:25:04.888025+0000","last_became_peered":"2024-01-21T22:25:04.888025+0000","last_unstale":"2024-01-21T22:25:07.304314+0000","last_undegraded":"2024-01-21T22:25:07.304314+0000","last_fullsized":"2024-01-21T22:25:07.304314+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:57:44.129490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297099+0000","last_change":"2024-01-21T22:25:04.711908+0000","last_active":"2024-01-21T22:25:07.297099+0000","last_peered":"2024-01-21T22:25:07.297099+0000","last_clean":"2024-01-21T22:25:07.297099+0000","last_became_active":"2024-01-21T22:25:04.711654+0000","last_became_peered":"2024-01-21T22:25:04.711654+0000","last_unstale":"2024-01-21T22:25:07.297099+0000","last_undegraded":"2024-01-21T22:25:07.297099+0000","last_fullsized":"2024-01-21T22:25:07.297099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:20:41.612265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093463+0000","last_change":"2024-01-21T22:25:04.876757+0000","last_active":"2024-01-21T22:25:07.093463+0000","last_peered":"2024-01-21T22:25:07.093463+0000","last_clean":"2024-01-21T22:25:07.093463+0000","last_became_active":"2024-01-21T22:25:04.876012+0000","last_became_peered":"2024-01-21T22:25:04.876012+0000","last_unstale":"2024-01-21T22:25:07.093463+0000","last_undegraded":"2024-01-21T22:25:07.093463+0000","last_fullsized":"2024-01-21T22:25:07.093463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:00:56.239773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.296179+0000","last_change":"2024-01-21T22:25:04.715277+0000","last_active":"2024-01-21T22:25:07.296179+0000","last_peered":"2024-01-21T22:25:07.296179+0000","last_clean":"2024-01-21T22:25:07.296179+0000","last_became_active":"2024-01-21T22:25:04.715034+0000","last_became_peered":"2024-01-21T22:25:04.715034+0000","last_unstale":"2024-01-21T22:25:07.296179+0000","last_undegraded":"2024-01-21T22:25:07.296179+0000","last_fullsized":"2024-01-21T22:25:07.296179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:55:28.069411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.143324+0000","last_change":"2024-01-21T22:25:04.844472+0000","last_active":"2024-01-21T22:25:07.143324+0000","last_peered":"2024-01-21T22:25:07.143324+0000","last_clean":"2024-01-21T22:25:07.143324+0000","last_became_active":"2024-01-21T22:25:04.843827+0000","last_became_peered":"2024-01-21T22:25:04.843827+0000","last_unstale":"2024-01-21T22:25:07.143324+0000","last_undegraded":"2024-01-21T22:25:07.143324+0000","last_fullsized":"2024-01-21T22:25:07.143324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:58:10.552987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.303755+0000","last_change":"2024-01-21T22:25:04.713764+0000","last_active":"2024-01-21T22:25:07.303755+0000","last_peered":"2024-01-21T22:25:07.303755+0000","last_clean":"2024-01-21T22:25:07.303755+0000","last_became_active":"2024-01-21T22:25:04.712814+0000","last_became_peered":"2024-01-21T22:25:04.712814+0000","last_unstale":"2024-01-21T22:25:07.303755+0000","last_undegraded":"2024-01-21T22:25:07.303755+0000","last_fullsized":"2024-01-21T22:25:07.303755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:16:41.440927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297178+0000","last_change":"2024-01-21T22:25:03.703079+0000","last_active":"2024-01-21T22:25:07.297178+0000","last_peered":"2024-01-21T22:25:07.297178+0000","last_clean":"2024-01-21T22:25:07.297178+0000","last_became_active":"2024-01-21T22:25:03.702776+0000","last_became_peered":"2024-01-21T22:25:03.702776+0000","last_unstale":"2024-01-21T22:25:07.297178+0000","last_undegraded":"2024-01-21T22:25:07.297178+0000","last_fullsized":"2024-01-21T22:25:07.297178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:34:13.302149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151769+0000","last_change":"2024-01-21T22:25:04.877158+0000","last_active":"2024-01-21T22:25:07.151769+0000","last_peered":"2024-01-21T22:25:07.151769+0000","last_clean":"2024-01-21T22:25:07.151769+0000","last_became_active":"2024-01-21T22:25:04.876920+0000","last_became_peered":"2024-01-21T22:25:04.876920+0000","last_unstale":"2024-01-21T22:25:07.151769+0000","last_undegraded":"2024-01-21T22:25:07.151769+0000","last_fullsized":"2024-01-21T22:25:07.151769+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:46:55.451173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093458+0000","last_change":"2024-01-21T22:25:03.720178+0000","last_active":"2024-01-21T22:25:07.093458+0000","last_peered":"2024-01-21T22:25:07.093458+0000","last_clean":"2024-01-21T22:25:07.093458+0000","last_became_active":"2024-01-21T22:25:03.719909+0000","last_became_peered":"2024-01-21T22:25:03.719909+0000","last_unstale":"2024-01-21T22:25:07.093458+0000","last_undegraded":"2024-01-21T22:25:07.093458+0000","last_fullsized":"2024-01-21T22:25:07.093458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:44:53.910872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.877462+0000","last_change":"2024-01-21T22:25:03.723005+0000","last_active":"2024-01-21T22:25:05.877462+0000","last_peered":"2024-01-21T22:25:05.877462+0000","last_clean":"2024-01-21T22:25:05.877462+0000","last_became_active":"2024-01-21T22:25:03.722758+0000","last_became_peered":"2024-01-21T22:25:03.722758+0000","last_unstale":"2024-01-21T22:25:05.877462+0000","last_undegraded":"2024-01-21T22:25:05.877462+0000","last_fullsized":"2024-01-21T22:25:05.877462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:16:40.192808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297215+0000","last_change":"2024-01-21T22:25:04.844030+0000","last_active":"2024-01-21T22:25:07.297215+0000","last_peered":"2024-01-21T22:25:07.297215+0000","last_clean":"2024-01-21T22:25:07.297215+0000","last_became_active":"2024-01-21T22:25:04.843714+0000","last_became_peered":"2024-01-21T22:25:04.843714+0000","last_unstale":"2024-01-21T22:25:07.297215+0000","last_undegraded":"2024-01-21T22:25:07.297215+0000","last_fullsized":"2024-01-21T22:25:07.297215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:25:50.380424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.521757+0000","last_change":"2024-01-21T22:25:03.702767+0000","last_active":"2024-01-21T22:25:05.521757+0000","last_peered":"2024-01-21T22:25:05.521757+0000","last_clean":"2024-01-21T22:25:05.521757+0000","last_became_active":"2024-01-21T22:25:03.702140+0000","last_became_peered":"2024-01-21T22:25:03.702140+0000","last_unstale":"2024-01-21T22:25:05.521757+0000","last_undegraded":"2024-01-21T22:25:05.521757+0000","last_fullsized":"2024-01-21T22:25:05.521757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:46:54.203055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151688+0000","last_change":"2024-01-21T22:25:04.867173+0000","last_active":"2024-01-21T22:25:07.151688+0000","last_peered":"2024-01-21T22:25:07.151688+0000","last_clean":"2024-01-21T22:25:07.151688+0000","last_became_active":"2024-01-21T22:25:04.866945+0000","last_became_peered":"2024-01-21T22:25:04.866945+0000","last_unstale":"2024-01-21T22:25:07.151688+0000","last_undegraded":"2024-01-21T22:25:07.151688+0000","last_fullsized":"2024-01-21T22:25:07.151688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:15:41.725132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.882129+0000","last_change":"2024-01-21T22:25:03.715128+0000","last_active":"2024-01-21T22:25:05.882129+0000","last_peered":"2024-01-21T22:25:05.882129+0000","last_clean":"2024-01-21T22:25:05.882129+0000","last_became_active":"2024-01-21T22:25:03.714039+0000","last_became_peered":"2024-01-21T22:25:03.714039+0000","last_unstale":"2024-01-21T22:25:05.882129+0000","last_undegraded":"2024-01-21T22:25:05.882129+0000","last_fullsized":"2024-01-21T22:25:05.882129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:09:09.988399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.144063+0000","last_change":"2024-01-21T22:25:04.886534+0000","last_active":"2024-01-21T22:25:07.144063+0000","last_peered":"2024-01-21T22:25:07.144063+0000","last_clean":"2024-01-21T22:25:07.144063+0000","last_became_active":"2024-01-21T22:25:04.885963+0000","last_became_peered":"2024-01-21T22:25:04.885963+0000","last_unstale":"2024-01-21T22:25:07.144063+0000","last_undegraded":"2024-01-21T22:25:07.144063+0000","last_fullsized":"2024-01-21T22:25:07.144063+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:44:31.867220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.610812+0000","last_change":"2024-01-21T22:25:03.718237+0000","last_active":"2024-01-21T22:25:05.610812+0000","last_peered":"2024-01-21T22:25:05.610812+0000","last_clean":"2024-01-21T22:25:05.610812+0000","last_became_active":"2024-01-21T22:25:03.717937+0000","last_became_peered":"2024-01-21T22:25:03.717937+0000","last_unstale":"2024-01-21T22:25:05.610812+0000","last_undegraded":"2024-01-21T22:25:05.610812+0000","last_fullsized":"2024-01-21T22:25:05.610812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:31:24.737692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298047+0000","last_change":"2024-01-21T22:25:04.882743+0000","last_active":"2024-01-21T22:25:07.298047+0000","last_peered":"2024-01-21T22:25:07.298047+0000","last_clean":"2024-01-21T22:25:07.298047+0000","last_became_active":"2024-01-21T22:25:04.882478+0000","last_became_peered":"2024-01-21T22:25:04.882478+0000","last_unstale":"2024-01-21T22:25:07.298047+0000","last_undegraded":"2024-01-21T22:25:07.298047+0000","last_fullsized":"2024-01-21T22:25:07.298047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:42:39.977125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.146198+0000","last_change":"2024-01-21T22:25:03.703672+0000","last_active":"2024-01-21T22:25:07.146198+0000","last_peered":"2024-01-21T22:25:07.146198+0000","last_clean":"2024-01-21T22:25:07.146198+0000","last_became_active":"2024-01-21T22:25:03.703426+0000","last_became_peered":"2024-01-21T22:25:03.703426+0000","last_unstale":"2024-01-21T22:25:07.146198+0000","last_undegraded":"2024-01-21T22:25:07.146198+0000","last_fullsized":"2024-01-21T22:25:07.146198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:41:51.793206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093763+0000","last_change":"2024-01-21T22:25:04.887306+0000","last_active":"2024-01-21T22:25:07.093763+0000","last_peered":"2024-01-21T22:25:07.093763+0000","last_clean":"2024-01-21T22:25:07.093763+0000","last_became_active":"2024-01-21T22:25:04.886970+0000","last_became_peered":"2024-01-21T22:25:04.886970+0000","last_unstale":"2024-01-21T22:25:07.093763+0000","last_undegraded":"2024-01-21T22:25:07.093763+0000","last_fullsized":"2024-01-21T22:25:07.093763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:59:45.239426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"23'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.158991+0000","last_change":"2024-01-21T22:25:03.714102+0000","last_active":"2024-01-21T22:25:07.158991+0000","last_peered":"2024-01-21T22:25:07.158991+0000","last_clean":"2024-01-21T22:25:07.158991+0000","last_became_active":"2024-01-21T22:25:03.713717+0000","last_became_peered":"2024-01-21T22:25:03.713717+0000","last_unstale":"2024-01-21T22:25:07.158991+0000","last_undegraded":"2024-01-21T22:25:07.158991+0000","last_fullsized":"2024-01-21T22:25:07.158991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:32:15.181238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286154+0000","last_change":"2024-01-21T22:25:04.881307+0000","last_active":"2024-01-21T22:25:07.286154+0000","last_peered":"2024-01-21T22:25:07.286154+0000","last_clean":"2024-01-21T22:25:07.286154+0000","last_became_active":"2024-01-21T22:25:04.880472+0000","last_became_peered":"2024-01-21T22:25:04.880472+0000","last_unstale":"2024-01-21T22:25:07.286154+0000","last_undegraded":"2024-01-21T22:25:07.286154+0000","last_fullsized":"2024-01-21T22:25:07.286154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:29:02.950903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151339+0000","last_change":"2024-01-21T22:25:03.721825+0000","last_active":"2024-01-21T22:25:07.151339+0000","last_peered":"2024-01-21T22:25:07.151339+0000","last_clean":"2024-01-21T22:25:07.151339+0000","last_became_active":"2024-01-21T22:25:03.721502+0000","last_became_peered":"2024-01-21T22:25:03.721502+0000","last_unstale":"2024-01-21T22:25:07.151339+0000","last_undegraded":"2024-01-21T22:25:07.151339+0000","last_fullsized":"2024-01-21T22:25:07.151339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:09:09.988399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304218+0000","last_change":"2024-01-21T22:25:04.877455+0000","last_active":"2024-01-21T22:25:07.304218+0000","last_peered":"2024-01-21T22:25:07.304218+0000","last_clean":"2024-01-21T22:25:07.304218+0000","last_became_active":"2024-01-21T22:25:04.877257+0000","last_became_peered":"2024-01-21T22:25:04.877257+0000","last_unstale":"2024-01-21T22:25:07.304218+0000","last_undegraded":"2024-01-21T22:25:07.304218+0000","last_fullsized":"2024-01-21T22:25:07.304218+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:34:14.550267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.522179+0000","last_change":"2024-01-21T22:25:03.706077+0000","last_active":"2024-01-21T22:25:05.522179+0000","last_peered":"2024-01-21T22:25:05.522179+0000","last_clean":"2024-01-21T22:25:05.522179+0000","last_became_active":"2024-01-21T22:25:03.702798+0000","last_became_peered":"2024-01-21T22:25:03.702798+0000","last_unstale":"2024-01-21T22:25:05.522179+0000","last_undegraded":"2024-01-21T22:25:05.522179+0000","last_fullsized":"2024-01-21T22:25:05.522179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:20:40.364147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093643+0000","last_change":"2024-01-21T22:25:04.890636+0000","last_active":"2024-01-21T22:25:07.093643+0000","last_peered":"2024-01-21T22:25:07.093643+0000","last_clean":"2024-01-21T22:25:07.093643+0000","last_became_active":"2024-01-21T22:25:04.890271+0000","last_became_peered":"2024-01-21T22:25:04.890271+0000","last_unstale":"2024-01-21T22:25:07.093643+0000","last_undegraded":"2024-01-21T22:25:07.093643+0000","last_fullsized":"2024-01-21T22:25:07.093643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:15:46.754395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.609220+0000","last_change":"2024-01-21T22:25:03.717202+0000","last_active":"2024-01-21T22:25:05.609220+0000","last_peered":"2024-01-21T22:25:05.609220+0000","last_clean":"2024-01-21T22:25:05.609220+0000","last_became_active":"2024-01-21T22:25:03.716846+0000","last_became_peered":"2024-01-21T22:25:03.716846+0000","last_unstale":"2024-01-21T22:25:05.609220+0000","last_undegraded":"2024-01-21T22:25:05.609220+0000","last_fullsized":"2024-01-21T22:25:05.609220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:10.904923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.303888+0000","last_change":"2024-01-21T22:25:04.715053+0000","last_active":"2024-01-21T22:25:07.303888+0000","last_peered":"2024-01-21T22:25:07.303888+0000","last_clean":"2024-01-21T22:25:07.303888+0000","last_became_active":"2024-01-21T22:25:04.713566+0000","last_became_peered":"2024-01-21T22:25:04.713566+0000","last_unstale":"2024-01-21T22:25:07.303888+0000","last_undegraded":"2024-01-21T22:25:07.303888+0000","last_fullsized":"2024-01-21T22:25:07.303888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:50:41.306582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"23'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304071+0000","last_change":"2024-01-21T22:25:03.703017+0000","last_active":"2024-01-21T22:25:07.304071+0000","last_peered":"2024-01-21T22:25:07.304071+0000","last_clean":"2024-01-21T22:25:07.304071+0000","last_became_active":"2024-01-21T22:25:03.702742+0000","last_became_peered":"2024-01-21T22:25:03.702742+0000","last_unstale":"2024-01-21T22:25:07.304071+0000","last_undegraded":"2024-01-21T22:25:07.304071+0000","last_fullsized":"2024-01-21T22:25:07.304071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:42:03.386594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093975+0000","last_change":"2024-01-21T22:25:04.876507+0000","last_active":"2024-01-21T22:25:07.093975+0000","last_peered":"2024-01-21T22:25:07.093975+0000","last_clean":"2024-01-21T22:25:07.093975+0000","last_became_active":"2024-01-21T22:25:04.876169+0000","last_became_peered":"2024-01-21T22:25:04.876169+0000","last_unstale":"2024-01-21T22:25:07.093975+0000","last_undegraded":"2024-01-21T22:25:07.093975+0000","last_fullsized":"2024-01-21T22:25:07.093975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:12:11.958115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.152598+0000","last_change":"2024-01-21T22:25:03.720257+0000","last_active":"2024-01-21T22:25:07.152598+0000","last_peered":"2024-01-21T22:25:07.152598+0000","last_clean":"2024-01-21T22:25:07.152598+0000","last_became_active":"2024-01-21T22:25:03.720009+0000","last_became_peered":"2024-01-21T22:25:03.720009+0000","last_unstale":"2024-01-21T22:25:07.152598+0000","last_undegraded":"2024-01-21T22:25:07.152598+0000","last_fullsized":"2024-01-21T22:25:07.152598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:37:56.569677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.144048+0000","last_change":"2024-01-21T22:25:04.883112+0000","last_active":"2024-01-21T22:25:07.144048+0000","last_peered":"2024-01-21T22:25:07.144048+0000","last_clean":"2024-01-21T22:25:07.144048+0000","last_became_active":"2024-01-21T22:25:04.882767+0000","last_became_peered":"2024-01-21T22:25:04.882767+0000","last_unstale":"2024-01-21T22:25:07.144048+0000","last_undegraded":"2024-01-21T22:25:07.144048+0000","last_fullsized":"2024-01-21T22:25:07.144048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:18:03.439738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.303803+0000","last_change":"2024-01-21T22:25:03.703516+0000","last_active":"2024-01-21T22:25:07.303803+0000","last_peered":"2024-01-21T22:25:07.303803+0000","last_clean":"2024-01-21T22:25:07.303803+0000","last_became_active":"2024-01-21T22:25:03.703256+0000","last_became_peered":"2024-01-21T22:25:03.703256+0000","last_unstale":"2024-01-21T22:25:07.303803+0000","last_undegraded":"2024-01-21T22:25:07.303803+0000","last_fullsized":"2024-01-21T22:25:07.303803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:57:42.881371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286834+0000","last_change":"2024-01-21T22:25:04.884129+0000","last_active":"2024-01-21T22:25:07.286834+0000","last_peered":"2024-01-21T22:25:07.286834+0000","last_clean":"2024-01-21T22:25:07.286834+0000","last_became_active":"2024-01-21T22:25:04.883839+0000","last_became_peered":"2024-01-21T22:25:04.883839+0000","last_unstale":"2024-01-21T22:25:07.286834+0000","last_undegraded":"2024-01-21T22:25:07.286834+0000","last_fullsized":"2024-01-21T22:25:07.286834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:43:55.049706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298049+0000","last_change":"2024-01-21T22:25:03.708366+0000","last_active":"2024-01-21T22:25:07.298049+0000","last_peered":"2024-01-21T22:25:07.298049+0000","last_clean":"2024-01-21T22:25:07.298049+0000","last_became_active":"2024-01-21T22:25:03.708101+0000","last_became_peered":"2024-01-21T22:25:03.708101+0000","last_unstale":"2024-01-21T22:25:07.298049+0000","last_undegraded":"2024-01-21T22:25:07.298049+0000","last_fullsized":"2024-01-21T22:25:07.298049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:57:42.881371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286513+0000","last_change":"2024-01-21T22:25:04.877993+0000","last_active":"2024-01-21T22:25:07.286513+0000","last_peered":"2024-01-21T22:25:07.286513+0000","last_clean":"2024-01-21T22:25:07.286513+0000","last_became_active":"2024-01-21T22:25:04.877661+0000","last_became_peered":"2024-01-21T22:25:04.877661+0000","last_unstale":"2024-01-21T22:25:07.286513+0000","last_undegraded":"2024-01-21T22:25:07.286513+0000","last_fullsized":"2024-01-21T22:25:07.286513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:20:40.947205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.878348+0000","last_change":"2024-01-21T22:25:03.721559+0000","last_active":"2024-01-21T22:25:05.878348+0000","last_peered":"2024-01-21T22:25:05.878348+0000","last_clean":"2024-01-21T22:25:05.878348+0000","last_became_active":"2024-01-21T22:25:03.721281+0000","last_became_peered":"2024-01-21T22:25:03.721281+0000","last_unstale":"2024-01-21T22:25:05.878348+0000","last_undegraded":"2024-01-21T22:25:05.878348+0000","last_fullsized":"2024-01-21T22:25:05.878348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:55:19.679388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.094279+0000","last_change":"2024-01-21T22:25:04.883087+0000","last_active":"2024-01-21T22:25:07.094279+0000","last_peered":"2024-01-21T22:25:07.094279+0000","last_clean":"2024-01-21T22:25:07.094279+0000","last_became_active":"2024-01-21T22:25:04.880227+0000","last_became_peered":"2024-01-21T22:25:04.880227+0000","last_unstale":"2024-01-21T22:25:07.094279+0000","last_undegraded":"2024-01-21T22:25:07.094279+0000","last_fullsized":"2024-01-21T22:25:07.094279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:58:09.653538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.880999+0000","last_change":"2024-01-21T22:24:52.968324+0000","last_active":"2024-01-21T22:25:05.880999+0000","last_peered":"2024-01-21T22:25:05.880999+0000","last_clean":"2024-01-21T22:25:05.880999+0000","last_became_active":"2024-01-21T22:24:52.967018+0000","last_became_peered":"2024-01-21T22:24:52.967018+0000","last_unstale":"2024-01-21T22:25:05.880999+0000","last_undegraded":"2024-01-21T22:25:05.880999+0000","last_fullsized":"2024-01-21T22:25:05.880999+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:24:50.312540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:24:50.312540+0000","last_clean_scrub_stamp":"2024-01-21T22:24:50.312540+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:52:04.800073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.522445+0000","last_change":"2024-01-21T22:25:03.707407+0000","last_active":"2024-01-21T22:25:05.522445+0000","last_peered":"2024-01-21T22:25:05.522445+0000","last_clean":"2024-01-21T22:25:05.522445+0000","last_became_active":"2024-01-21T22:25:03.707085+0000","last_became_peered":"2024-01-21T22:25:03.707085+0000","last_unstale":"2024-01-21T22:25:05.522445+0000","last_undegraded":"2024-01-21T22:25:05.522445+0000","last_fullsized":"2024-01-21T22:25:05.522445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:44:22.501512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151755+0000","last_change":"2024-01-21T22:25:04.868564+0000","last_active":"2024-01-21T22:25:07.151755+0000","last_peered":"2024-01-21T22:25:07.151755+0000","last_clean":"2024-01-21T22:25:07.151755+0000","last_became_active":"2024-01-21T22:25:04.868247+0000","last_became_peered":"2024-01-21T22:25:04.868247+0000","last_unstale":"2024-01-21T22:25:07.151755+0000","last_undegraded":"2024-01-21T22:25:07.151755+0000","last_fullsized":"2024-01-21T22:25:07.151755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:30:50.039501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.521811+0000","last_change":"2024-01-21T22:25:03.706070+0000","last_active":"2024-01-21T22:25:05.521811+0000","last_peered":"2024-01-21T22:25:05.521811+0000","last_clean":"2024-01-21T22:25:05.521811+0000","last_became_active":"2024-01-21T22:25:03.705785+0000","last_became_peered":"2024-01-21T22:25:03.705785+0000","last_unstale":"2024-01-21T22:25:05.521811+0000","last_undegraded":"2024-01-21T22:25:05.521811+0000","last_fullsized":"2024-01-21T22:25:05.521811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:30.929444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304132+0000","last_change":"2024-01-21T22:25:04.845436+0000","last_active":"2024-01-21T22:25:07.304132+0000","last_peered":"2024-01-21T22:25:07.304132+0000","last_clean":"2024-01-21T22:25:07.304132+0000","last_became_active":"2024-01-21T22:25:04.845274+0000","last_became_peered":"2024-01-21T22:25:04.845274+0000","last_unstale":"2024-01-21T22:25:07.304132+0000","last_undegraded":"2024-01-21T22:25:07.304132+0000","last_fullsized":"2024-01-21T22:25:07.304132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:08:36.686787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.878258+0000","last_change":"2024-01-21T22:25:03.709174+0000","last_active":"2024-01-21T22:25:05.878258+0000","last_peered":"2024-01-21T22:25:05.878258+0000","last_clean":"2024-01-21T22:25:05.878258+0000","last_became_active":"2024-01-21T22:25:03.708894+0000","last_became_peered":"2024-01-21T22:25:03.708894+0000","last_unstale":"2024-01-21T22:25:05.878258+0000","last_undegraded":"2024-01-21T22:25:05.878258+0000","last_fullsized":"2024-01-21T22:25:05.878258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:19:26.591140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297083+0000","last_change":"2024-01-21T22:25:04.711947+0000","last_active":"2024-01-21T22:25:07.297083+0000","last_peered":"2024-01-21T22:25:07.297083+0000","last_clean":"2024-01-21T22:25:07.297083+0000","last_became_active":"2024-01-21T22:25:04.711694+0000","last_became_peered":"2024-01-21T22:25:04.711694+0000","last_unstale":"2024-01-21T22:25:07.297083+0000","last_undegraded":"2024-01-21T22:25:07.297083+0000","last_fullsized":"2024-01-21T22:25:07.297083+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:37:57.817795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.522504+0000","last_change":"2024-01-21T22:25:03.706355+0000","last_active":"2024-01-21T22:25:05.522504+0000","last_peered":"2024-01-21T22:25:05.522504+0000","last_clean":"2024-01-21T22:25:05.522504+0000","last_became_active":"2024-01-21T22:25:03.703876+0000","last_became_peered":"2024-01-21T22:25:03.703876+0000","last_unstale":"2024-01-21T22:25:05.522504+0000","last_undegraded":"2024-01-21T22:25:05.522504+0000","last_fullsized":"2024-01-21T22:25:05.522504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:42:38.729006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.285835+0000","last_change":"2024-01-21T22:25:04.885997+0000","last_active":"2024-01-21T22:25:07.285835+0000","last_peered":"2024-01-21T22:25:07.285835+0000","last_clean":"2024-01-21T22:25:07.285835+0000","last_became_active":"2024-01-21T22:25:04.884162+0000","last_became_peered":"2024-01-21T22:25:04.884162+0000","last_unstale":"2024-01-21T22:25:07.285835+0000","last_undegraded":"2024-01-21T22:25:07.285835+0000","last_fullsized":"2024-01-21T22:25:07.285835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:44:55.158990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.521383+0000","last_change":"2024-01-21T22:25:03.708300+0000","last_active":"2024-01-21T22:25:05.521383+0000","last_peered":"2024-01-21T22:25:05.521383+0000","last_clean":"2024-01-21T22:25:05.521383+0000","last_became_active":"2024-01-21T22:25:03.708036+0000","last_became_peered":"2024-01-21T22:25:03.708036+0000","last_unstale":"2024-01-21T22:25:05.521383+0000","last_undegraded":"2024-01-21T22:25:05.521383+0000","last_fullsized":"2024-01-21T22:25:05.521383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:23:51.196159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.143828+0000","last_change":"2024-01-21T22:25:04.844713+0000","last_active":"2024-01-21T22:25:07.143828+0000","last_peered":"2024-01-21T22:25:07.143828+0000","last_clean":"2024-01-21T22:25:07.143828+0000","last_became_active":"2024-01-21T22:25:04.844008+0000","last_became_peered":"2024-01-21T22:25:04.844008+0000","last_unstale":"2024-01-21T22:25:07.143828+0000","last_undegraded":"2024-01-21T22:25:07.143828+0000","last_fullsized":"2024-01-21T22:25:07.143828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:33:25.760064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.523113+0000","last_change":"2024-01-21T22:25:03.699093+0000","last_active":"2024-01-21T22:25:05.523113+0000","last_peered":"2024-01-21T22:25:05.523113+0000","last_clean":"2024-01-21T22:25:05.523113+0000","last_became_active":"2024-01-21T22:25:03.698443+0000","last_became_peered":"2024-01-21T22:25:03.698443+0000","last_unstale":"2024-01-21T22:25:05.523113+0000","last_undegraded":"2024-01-21T22:25:05.523113+0000","last_fullsized":"2024-01-21T22:25:05.523113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:20:39.699087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304411+0000","last_change":"2024-01-21T22:25:04.842733+0000","last_active":"2024-01-21T22:25:07.304411+0000","last_peered":"2024-01-21T22:25:07.304411+0000","last_clean":"2024-01-21T22:25:07.304411+0000","last_became_active":"2024-01-21T22:25:04.842479+0000","last_became_peered":"2024-01-21T22:25:04.842479+0000","last_unstale":"2024-01-21T22:25:07.304411+0000","last_undegraded":"2024-01-21T22:25:07.304411+0000","last_fullsized":"2024-01-21T22:25:07.304411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:51:06.931790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151980+0000","last_change":"2024-01-21T22:25:03.720306+0000","last_active":"2024-01-21T22:25:07.151980+0000","last_peered":"2024-01-21T22:25:07.151980+0000","last_clean":"2024-01-21T22:25:07.151980+0000","last_became_active":"2024-01-21T22:25:03.720067+0000","last_became_peered":"2024-01-21T22:25:03.720067+0000","last_unstale":"2024-01-21T22:25:07.151980+0000","last_undegraded":"2024-01-21T22:25:07.151980+0000","last_fullsized":"2024-01-21T22:25:07.151980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:52:22.985590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151977+0000","last_change":"2024-01-21T22:25:04.859628+0000","last_active":"2024-01-21T22:25:07.151977+0000","last_peered":"2024-01-21T22:25:07.151977+0000","last_clean":"2024-01-21T22:25:07.151977+0000","last_became_active":"2024-01-21T22:25:04.859270+0000","last_became_peered":"2024-01-21T22:25:04.859270+0000","last_unstale":"2024-01-21T22:25:07.151977+0000","last_undegraded":"2024-01-21T22:25:07.151977+0000","last_fullsized":"2024-01-21T22:25:07.151977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:56:24.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.282102+0000","last_change":"2024-01-21T22:25:03.722006+0000","last_active":"2024-01-21T22:25:07.282102+0000","last_peered":"2024-01-21T22:25:07.282102+0000","last_clean":"2024-01-21T22:25:07.282102+0000","last_became_active":"2024-01-21T22:25:03.721275+0000","last_became_peered":"2024-01-21T22:25:03.721275+0000","last_unstale":"2024-01-21T22:25:07.282102+0000","last_undegraded":"2024-01-21T22:25:07.282102+0000","last_fullsized":"2024-01-21T22:25:07.282102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:43:53.801587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151916+0000","last_change":"2024-01-21T22:25:04.858864+0000","last_active":"2024-01-21T22:25:07.151916+0000","last_peered":"2024-01-21T22:25:07.151916+0000","last_clean":"2024-01-21T22:25:07.151916+0000","last_became_active":"2024-01-21T22:25:04.858586+0000","last_became_peered":"2024-01-21T22:25:04.858586+0000","last_unstale":"2024-01-21T22:25:07.151916+0000","last_undegraded":"2024-01-21T22:25:07.151916+0000","last_fullsized":"2024-01-21T22:25:07.151916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:32.177562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.609941+0000","last_change":"2024-01-21T22:25:03.718436+0000","last_active":"2024-01-21T22:25:05.609941+0000","last_peered":"2024-01-21T22:25:05.609941+0000","last_clean":"2024-01-21T22:25:05.609941+0000","last_became_active":"2024-01-21T22:25:03.718171+0000","last_became_peered":"2024-01-21T22:25:03.718171+0000","last_unstale":"2024-01-21T22:25:05.609941+0000","last_undegraded":"2024-01-21T22:25:05.609941+0000","last_fullsized":"2024-01-21T22:25:05.609941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:05:40.702113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093711+0000","last_change":"2024-01-21T22:25:04.875736+0000","last_active":"2024-01-21T22:25:07.093711+0000","last_peered":"2024-01-21T22:25:07.093711+0000","last_clean":"2024-01-21T22:25:07.093711+0000","last_became_active":"2024-01-21T22:25:04.875368+0000","last_became_peered":"2024-01-21T22:25:04.875368+0000","last_unstale":"2024-01-21T22:25:07.093711+0000","last_undegraded":"2024-01-21T22:25:07.093711+0000","last_fullsized":"2024-01-21T22:25:07.093711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:39:31.273250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.877906+0000","last_change":"2024-01-21T22:25:03.722820+0000","last_active":"2024-01-21T22:25:05.877906+0000","last_peered":"2024-01-21T22:25:05.877906+0000","last_clean":"2024-01-21T22:25:05.877906+0000","last_became_active":"2024-01-21T22:25:03.722525+0000","last_became_peered":"2024-01-21T22:25:03.722525+0000","last_unstale":"2024-01-21T22:25:05.877906+0000","last_undegraded":"2024-01-21T22:25:05.877906+0000","last_fullsized":"2024-01-21T22:25:05.877906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:40:04.306585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151604+0000","last_change":"2024-01-21T22:25:04.857451+0000","last_active":"2024-01-21T22:25:07.151604+0000","last_peered":"2024-01-21T22:25:07.151604+0000","last_clean":"2024-01-21T22:25:07.151604+0000","last_became_active":"2024-01-21T22:25:04.857205+0000","last_became_peered":"2024-01-21T22:25:04.857205+0000","last_unstale":"2024-01-21T22:25:07.151604+0000","last_undegraded":"2024-01-21T22:25:07.151604+0000","last_fullsized":"2024-01-21T22:25:07.151604+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:12.153042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.877746+0000","last_change":"2024-01-21T22:25:03.708574+0000","last_active":"2024-01-21T22:25:05.877746+0000","last_peered":"2024-01-21T22:25:05.877746+0000","last_clean":"2024-01-21T22:25:05.877746+0000","last_became_active":"2024-01-21T22:25:03.708318+0000","last_became_peered":"2024-01-21T22:25:03.708318+0000","last_unstale":"2024-01-21T22:25:05.877746+0000","last_undegraded":"2024-01-21T22:25:05.877746+0000","last_fullsized":"2024-01-21T22:25:05.877746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:50:40.058463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.296286+0000","last_change":"2024-01-21T22:25:04.843079+0000","last_active":"2024-01-21T22:25:07.296286+0000","last_peered":"2024-01-21T22:25:07.296286+0000","last_clean":"2024-01-21T22:25:07.296286+0000","last_became_active":"2024-01-21T22:25:04.842815+0000","last_became_peered":"2024-01-21T22:25:04.842815+0000","last_unstale":"2024-01-21T22:25:07.296286+0000","last_undegraded":"2024-01-21T22:25:07.296286+0000","last_fullsized":"2024-01-21T22:25:07.296286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:16:41.440927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.281616+0000","last_change":"2024-01-21T22:25:03.710506+0000","last_active":"2024-01-21T22:25:07.281616+0000","last_peered":"2024-01-21T22:25:07.281616+0000","last_clean":"2024-01-21T22:25:07.281616+0000","last_became_active":"2024-01-21T22:25:03.710262+0000","last_became_peered":"2024-01-21T22:25:03.710262+0000","last_unstale":"2024-01-21T22:25:07.281616+0000","last_undegraded":"2024-01-21T22:25:07.281616+0000","last_fullsized":"2024-01-21T22:25:07.281616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:25:49.132306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093352+0000","last_change":"2024-01-21T22:25:04.876469+0000","last_active":"2024-01-21T22:25:07.093352+0000","last_peered":"2024-01-21T22:25:07.093352+0000","last_clean":"2024-01-21T22:25:07.093352+0000","last_became_active":"2024-01-21T22:25:04.875619+0000","last_became_peered":"2024-01-21T22:25:04.875619+0000","last_unstale":"2024-01-21T22:25:07.093352+0000","last_undegraded":"2024-01-21T22:25:07.093352+0000","last_fullsized":"2024-01-21T22:25:07.093352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:30:20.167607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.094159+0000","last_change":"2024-01-21T22:25:04.891067+0000","last_active":"2024-01-21T22:25:07.094159+0000","last_peered":"2024-01-21T22:25:07.094159+0000","last_clean":"2024-01-21T22:25:07.094159+0000","last_became_active":"2024-01-21T22:25:04.890515+0000","last_became_peered":"2024-01-21T22:25:04.890515+0000","last_unstale":"2024-01-21T22:25:07.094159+0000","last_undegraded":"2024-01-21T22:25:07.094159+0000","last_fullsized":"2024-01-21T22:25:07.094159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:25:48.726591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.158438+0000","last_change":"2024-01-21T22:25:03.719412+0000","last_active":"2024-01-21T22:25:07.158438+0000","last_peered":"2024-01-21T22:25:07.158438+0000","last_clean":"2024-01-21T22:25:07.158438+0000","last_became_active":"2024-01-21T22:25:03.718163+0000","last_became_peered":"2024-01-21T22:25:03.718163+0000","last_unstale":"2024-01-21T22:25:07.158438+0000","last_undegraded":"2024-01-21T22:25:07.158438+0000","last_fullsized":"2024-01-21T22:25:07.158438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:56:22.817420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304192+0000","last_change":"2024-01-21T22:25:04.889166+0000","last_active":"2024-01-21T22:25:07.304192+0000","last_peered":"2024-01-21T22:25:07.304192+0000","last_clean":"2024-01-21T22:25:07.304192+0000","last_became_active":"2024-01-21T22:25:04.888910+0000","last_became_peered":"2024-01-21T22:25:04.888910+0000","last_unstale":"2024-01-21T22:25:07.304192+0000","last_undegraded":"2024-01-21T22:25:07.304192+0000","last_fullsized":"2024-01-21T22:25:07.304192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:55:53.892439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.156719+0000","last_change":"2024-01-21T22:25:03.718932+0000","last_active":"2024-01-21T22:25:07.156719+0000","last_peered":"2024-01-21T22:25:07.156719+0000","last_clean":"2024-01-21T22:25:07.156719+0000","last_became_active":"2024-01-21T22:25:03.718649+0000","last_became_peered":"2024-01-21T22:25:03.718649+0000","last_unstale":"2024-01-21T22:25:07.156719+0000","last_undegraded":"2024-01-21T22:25:07.156719+0000","last_fullsized":"2024-01-21T22:25:07.156719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:30.929444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297922+0000","last_change":"2024-01-21T22:25:04.715429+0000","last_active":"2024-01-21T22:25:07.297922+0000","last_peered":"2024-01-21T22:25:07.297922+0000","last_clean":"2024-01-21T22:25:07.297922+0000","last_became_active":"2024-01-21T22:25:04.714720+0000","last_became_peered":"2024-01-21T22:25:04.714720+0000","last_unstale":"2024-01-21T22:25:07.297922+0000","last_undegraded":"2024-01-21T22:25:07.297922+0000","last_fullsized":"2024-01-21T22:25:07.297922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:44:23.749631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298298+0000","last_change":"2024-01-21T22:25:03.703289+0000","last_active":"2024-01-21T22:25:07.298298+0000","last_peered":"2024-01-21T22:25:07.298298+0000","last_clean":"2024-01-21T22:25:07.298298+0000","last_became_active":"2024-01-21T22:25:03.703020+0000","last_became_peered":"2024-01-21T22:25:03.703020+0000","last_unstale":"2024-01-21T22:25:07.298298+0000","last_undegraded":"2024-01-21T22:25:07.298298+0000","last_fullsized":"2024-01-21T22:25:07.298298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:10.904923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298304+0000","last_change":"2024-01-21T22:25:04.715894+0000","last_active":"2024-01-21T22:25:07.298304+0000","last_peered":"2024-01-21T22:25:07.298304+0000","last_clean":"2024-01-21T22:25:07.298304+0000","last_became_active":"2024-01-21T22:25:04.715652+0000","last_became_peered":"2024-01-21T22:25:04.715652+0000","last_unstale":"2024-01-21T22:25:07.298304+0000","last_undegraded":"2024-01-21T22:25:07.298304+0000","last_fullsized":"2024-01-21T22:25:07.298304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:13:04.601911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"23'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.303733+0000","last_change":"2024-01-21T22:25:03.702339+0000","last_active":"2024-01-21T22:25:07.303733+0000","last_peered":"2024-01-21T22:25:07.303733+0000","last_clean":"2024-01-21T22:25:07.303733+0000","last_became_active":"2024-01-21T22:25:03.702087+0000","last_became_peered":"2024-01-21T22:25:03.702087+0000","last_unstale":"2024-01-21T22:25:07.303733+0000","last_undegraded":"2024-01-21T22:25:07.303733+0000","last_fullsized":"2024-01-21T22:25:07.303733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:51:05.683671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":245760,"data_stored":85152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]}]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":85028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":839680,"data_stored":675488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":87312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542154,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":85610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.179}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.05}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.304}]}]},{"osd":2,"up_from":14,"seq":60129542152,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":249856,"data_stored":86640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.371}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5720000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.099}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":835584,"data_stored":675466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.389}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2609999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]}]},{"osd":0,"up_from":14,"seq":60129542154,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":245760,"data_stored":85084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7130000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4370000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-21T22:25:16.338 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-01-21T22:25:16.338 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-01-21T22:25:16.338 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-01-21T22:25:16.338 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-01-21T22:25:16.339 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-01-21T22:25:16.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.593+0000 7f4295000640 1 Processor -- start 2024-01-21T22:25:16.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.593+0000 7f4295000640 1 -- start start 2024-01-21T22:25:16.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.595+0000 7f4295000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f429015a680 0x7f42901654b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:16.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.595+0000 7f4295000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42900c61c0 con 0x7f429015a250 2024-01-21T22:25:16.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.595+0000 7f4295000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4290159dc0 con 0x7f4290153cc0 2024-01-21T22:25:16.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.595+0000 7f4295000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4290159b20 con 0x7f429015a680 2024-01-21T22:25:16.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.596+0000 7f428e800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f429015a680 0x7f42901654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:16.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.596+0000 7f428d400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f429015a250 0x7f429006a6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:60688/0 (socket says 172.21.15.138:60688) 2024-01-21T22:25:16.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.596+0000 7f428d400640 1 -- 172.21.15.138:0/301222365 learned_addr learned my addr 172.21.15.138:0/301222365 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:16.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.598+0000 7f428ca00640 1 -- 172.21.15.138:0/301222365 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3667855109 0 0) 0x7f42900c61c0 con 0x7f429015a250 2024-01-21T22:25:16.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.598+0000 7f428ca00640 1 -- 172.21.15.138:0/301222365 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4270003520 con 0x7f429015a250 2024-01-21T22:25:16.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.599+0000 7f428e800640 1 -- 172.21.15.138:0/301222365 >> v1:172.21.15.152:6789/0 conn(0x7f4290153cc0 legacy=0x7f42900c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:16.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.599+0000 7f428e800640 1 -- 172.21.15.138:0/301222365 >> v1:172.21.15.138:6789/0 conn(0x7f429015a250 legacy=0x7f429006a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:16.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.599+0000 7f428e800640 1 -- 172.21.15.138:0/301222365 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4290071270 con 0x7f429015a680 2024-01-21T22:25:16.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.603+0000 7f428e800640 1 --2- 172.21.15.138:0/301222365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f429015a680 0x7f42901654b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f42880097a0 tx=0x7f428802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddd82d343fa262b9 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:16.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.607+0000 7f428ca00640 1 -- 172.21.15.138:0/301222365 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4288030610 con 0x7f429015a680 2024-01-21T22:25:16.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.607+0000 7f428ca00640 1 -- 172.21.15.138:0/301222365 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4288030770 con 0x7f429015a680 2024-01-21T22:25:16.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.607+0000 7f428ca00640 1 -- 172.21.15.138:0/301222365 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4288030b00 con 0x7f429015a680 2024-01-21T22:25:16.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.608+0000 7f4295000640 1 -- 172.21.15.138:0/301222365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f429015a680 msgr2=0x7f42901654b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:16.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.608+0000 7f4295000640 1 --2- 172.21.15.138:0/301222365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f429015a680 0x7f42901654b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f42880097a0 tx=0x7f428802b960 comp rx=0 tx=0).stop 2024-01-21T22:25:16.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.608+0000 7f4295000640 1 -- 172.21.15.138:0/301222365 shutdown_connections 2024-01-21T22:25:16.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.608+0000 7f4295000640 1 --2- 172.21.15.138:0/301222365 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f429015a680 0x7f42901654b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:16.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.608+0000 7f4295000640 1 -- 172.21.15.138:0/301222365 >> 172.21.15.138:0/301222365 conn(0x7f42900cb0b0 msgr2=0x7f42900cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:16.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.608+0000 7f4295000640 1 -- 172.21.15.138:0/301222365 shutdown_connections 2024-01-21T22:25:16.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.608+0000 7f4295000640 1 -- 172.21.15.138:0/301222365 wait complete. 2024-01-21T22:25:16.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.610+0000 7f4295000640 1 Processor -- start 2024-01-21T22:25:16.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.610+0000 7f4295000640 1 -- start start 2024-01-21T22:25:16.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.611+0000 7f4295000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4290153cc0 0x7f429015da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:16.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.611+0000 7f4295000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f429015a250 0x7f42900ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:16.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.611+0000 7f4295000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f429015a680 0x7f42901f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:16.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.611+0000 7f4295000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429015b040 con 0x7f429015a250 2024-01-21T22:25:16.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.611+0000 7f4295000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429014e330 con 0x7f429015a680 2024-01-21T22:25:16.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.611+0000 7f4295000640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f429014e610 con 0x7f4290153cc0 2024-01-21T22:25:16.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.612+0000 7f428d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f429015a250 0x7f42900ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:16.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.612+0000 7f428e800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f429015a680 0x7f42901f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:16.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.612+0000 7f428de00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4290153cc0 0x7f429015da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:16.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.612+0000 7f428d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f429015a250 0x7f42900ca710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43568/0 (socket says 172.21.15.138:43568) 2024-01-21T22:25:16.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.612+0000 7f428d400640 1 -- 172.21.15.138:0/1769730693 learned_addr learned my addr 172.21.15.138:0/1769730693 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:16.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.616+0000 7f428de00640 1 -- 172.21.15.138:0/1769730693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f429015a680 msgr2=0x7f42901f2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:16.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.616+0000 7f428de00640 1 --2- 172.21.15.138:0/1769730693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f429015a680 0x7f42901f2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:16.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.616+0000 7f428de00640 1 -- 172.21.15.138:0/1769730693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f429015a250 msgr2=0x7f42900ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:16.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.616+0000 7f428de00640 1 --2- 172.21.15.138:0/1769730693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f429015a250 0x7f42900ca710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:16.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.616+0000 7f428de00640 1 -- 172.21.15.138:0/1769730693 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4288009400 con 0x7f4290153cc0 2024-01-21T22:25:16.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.618+0000 7f428de00640 1 --2- 172.21.15.138:0/1769730693 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4290153cc0 0x7f429015da50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f428000c340 tx=0x7f428000c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:16.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.622+0000 7f4286000640 1 -- 172.21.15.138:0/1769730693 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42800052d0 con 0x7f4290153cc0 2024-01-21T22:25:16.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.622+0000 7f4286000640 1 -- 172.21.15.138:0/1769730693 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4280005450 con 0x7f4290153cc0 2024-01-21T22:25:16.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.622+0000 7f4286000640 1 -- 172.21.15.138:0/1769730693 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280007260 con 0x7f4290153cc0 2024-01-21T22:25:16.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.622+0000 7f4295000640 1 -- 172.21.15.138:0/1769730693 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f429014e8f0 con 0x7f4290153cc0 2024-01-21T22:25:16.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.622+0000 7f4295000640 1 -- 172.21.15.138:0/1769730693 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f429015ace0 con 0x7f4290153cc0 2024-01-21T22:25:16.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.623+0000 7f4295000640 1 -- 172.21.15.138:0/1769730693 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f429004eaa0 con 0x7f4290153cc0 2024-01-21T22:25:16.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.642+0000 7f4286000640 1 -- 172.21.15.138:0/1769730693 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f42800073c0 con 0x7f4290153cc0 2024-01-21T22:25:16.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.643+0000 7f4286000640 1 --2- 172.21.15.138:0/1769730693 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f426806b380 0x7f426806d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:16.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.643+0000 7f428d400640 1 --2- 172.21.15.138:0/1769730693 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f426806b380 0x7f426806d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:16.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.644+0000 7f428d400640 1 --2- 172.21.15.138:0/1769730693 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f426806b380 0x7f426806d840 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f429015da00 tx=0x7f4278009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:16.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.645+0000 7f4286000640 1 -- 172.21.15.138:0/1769730693 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4280057db0 con 0x7f4290153cc0 2024-01-21T22:25:16.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:16.879+0000 7f4286000640 1 -- 172.21.15.138:0/1769730693 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f42800540f0 con 0x7f4290153cc0 2024-01-21T22:25:17.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.132+0000 7f4295000640 1 -- 172.21.15.138:0/1769730693 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f429006d7d0 con 0x7f4290153cc0 2024-01-21T22:25:17.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.272+0000 7f4286000640 1 -- 172.21.15.138:0/1769730693 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171311 (secure 0 0 0) 0x7f4280020090 con 0x7f4290153cc0 2024-01-21T22:25:17.275 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:25:17.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.277+0000 7f4295000640 1 -- 172.21.15.138:0/1769730693 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f426806b380 msgr2=0x7f426806d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:17.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.277+0000 7f4295000640 1 --2- 172.21.15.138:0/1769730693 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f426806b380 0x7f426806d840 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f429015da00 tx=0x7f4278009040 comp rx=0 tx=0).stop 2024-01-21T22:25:17.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.277+0000 7f4295000640 1 -- 172.21.15.138:0/1769730693 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4290153cc0 msgr2=0x7f429015da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:17.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.277+0000 7f4295000640 1 --2- 172.21.15.138:0/1769730693 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4290153cc0 0x7f429015da50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f428000c340 tx=0x7f428000c800 comp rx=0 tx=0).stop 2024-01-21T22:25:17.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.278+0000 7f4295000640 1 -- 172.21.15.138:0/1769730693 shutdown_connections 2024-01-21T22:25:17.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.278+0000 7f4295000640 1 --2- 172.21.15.138:0/1769730693 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f426806b380 0x7f426806d840 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:17.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.278+0000 7f4295000640 1 --2- 172.21.15.138:0/1769730693 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f429015a680 0x7f42901f2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:17.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.278+0000 7f4295000640 1 --2- 172.21.15.138:0/1769730693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f429015a250 0x7f42900ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:17.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.278+0000 7f4295000640 1 --2- 172.21.15.138:0/1769730693 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4290153cc0 0x7f429015da50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:17.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.278+0000 7f4295000640 1 -- 172.21.15.138:0/1769730693 >> 172.21.15.138:0/1769730693 conn(0x7f42900cb0b0 msgr2=0x7f42900c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:17.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.278+0000 7f4295000640 1 -- 172.21.15.138:0/1769730693 shutdown_connections 2024-01-21T22:25:17.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.278+0000 7f4295000640 1 -- 172.21.15.138:0/1769730693 wait complete. 2024-01-21T22:25:17.306 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":5,"active_gid":4111,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6800","nonce":314625564},{"type":"v1","addr":"172.21.15.152:6801","nonce":314625564}]},"active_addr":"172.21.15.152:6801/314625564","active_change":"2024-01-21T22:24:34.814649+0000","active_mgr_features":4540701272860131327,"available":true,"standbys":[{"gid":4108,"name":"y","mgr_features":4540701272860131327,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.152:0","nonce":4134574936}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.152:0","nonce":1385099247}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.152:0","nonce":2843129761}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.152:0","nonce":1770062691}]}]} 2024-01-21T22:25:17.310 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-01-21T22:25:17.310 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-01-21T22:25:17.310 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-21T22:25:17.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.513+0000 7f4667800640 1 Processor -- start 2024-01-21T22:25:17.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.513+0000 7f4667800640 1 -- start start 2024-01-21T22:25:17.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.514+0000 7f4667800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4660153f00 0x7f46600c4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:17.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.515+0000 7f4667800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f466006af50 con 0x7f46600cb580 2024-01-21T22:25:17.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.515+0000 7f4667800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f466006b230 con 0x7f466006c660 2024-01-21T22:25:17.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.515+0000 7f4667800640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466015aaa0 con 0x7f4660153f00 2024-01-21T22:25:17.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.515+0000 7f4664e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4660153f00 0x7f46600c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:17.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.515+0000 7f4665800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f466006c660 0x7f46600cc2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:58944/0 (socket says 172.21.15.138:58944) 2024-01-21T22:25:17.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.515+0000 7f4665800640 1 -- 172.21.15.138:0/1715866123 learned_addr learned my addr 172.21.15.138:0/1715866123 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:17.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.518+0000 7f4664e00640 1 -- 172.21.15.138:0/1715866123 >> v1:172.21.15.152:6789/0 conn(0x7f466006c660 legacy=0x7f46600cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:17.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.518+0000 7f4664e00640 1 -- 172.21.15.138:0/1715866123 >> v1:172.21.15.138:6789/0 conn(0x7f46600cb580 legacy=0x7f466006b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:17.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.518+0000 7f4664e00640 1 -- 172.21.15.138:0/1715866123 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46600c4950 con 0x7f4660153f00 2024-01-21T22:25:17.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.521+0000 7f4664e00640 1 --2- 172.21.15.138:0/1715866123 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4660153f00 0x7f46600c4320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f46500097d0 tx=0x7f465002b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c84a0549c6443aa server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:17.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.524+0000 7f465f400640 1 -- 172.21.15.138:0/1715866123 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4650031800 con 0x7f4660153f00 2024-01-21T22:25:17.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.524+0000 7f465f400640 1 -- 172.21.15.138:0/1715866123 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4650031960 con 0x7f4660153f00 2024-01-21T22:25:17.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.524+0000 7f465f400640 1 -- 172.21.15.138:0/1715866123 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4650031cf0 con 0x7f4660153f00 2024-01-21T22:25:17.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.525+0000 7f4667800640 1 -- 172.21.15.138:0/1715866123 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4660153f00 msgr2=0x7f46600c4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:17.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.525+0000 7f4667800640 1 --2- 172.21.15.138:0/1715866123 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4660153f00 0x7f46600c4320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f46500097d0 tx=0x7f465002b8a0 comp rx=0 tx=0).stop 2024-01-21T22:25:17.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.525+0000 7f4667800640 1 -- 172.21.15.138:0/1715866123 shutdown_connections 2024-01-21T22:25:17.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.525+0000 7f4667800640 1 --2- 172.21.15.138:0/1715866123 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4660153f00 0x7f46600c4320 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:17.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.525+0000 7f4667800640 1 -- 172.21.15.138:0/1715866123 >> 172.21.15.138:0/1715866123 conn(0x7f4660066820 msgr2=0x7f4660066c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:17.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.525+0000 7f4667800640 1 -- 172.21.15.138:0/1715866123 shutdown_connections 2024-01-21T22:25:17.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.525+0000 7f4667800640 1 -- 172.21.15.138:0/1715866123 wait complete. 2024-01-21T22:25:17.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.527+0000 7f4667800640 1 Processor -- start 2024-01-21T22:25:17.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.527+0000 7f4667800640 1 -- start start 2024-01-21T22:25:17.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.528+0000 7f4667800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f466006c660 0x7f466015abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:17.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.528+0000 7f4667800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46600cb580 0x7f4660149b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:17.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.528+0000 7f4667800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4660153f00 0x7f466014a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:17.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.528+0000 7f4667800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466015c900 con 0x7f466006c660 2024-01-21T22:25:17.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.528+0000 7f4667800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466014a590 con 0x7f46600cb580 2024-01-21T22:25:17.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.528+0000 7f4667800640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46600c54f0 con 0x7f4660153f00 2024-01-21T22:25:17.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.529+0000 7f465fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46600cb580 0x7f4660149b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:17.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.529+0000 7f4665800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4660153f00 0x7f466014a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:17.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.529+0000 7f465fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46600cb580 0x7f4660149b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:47866/0 (socket says 172.21.15.138:47866) 2024-01-21T22:25:17.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.529+0000 7f465fe00640 1 -- 172.21.15.138:0/3057415975 learned_addr learned my addr 172.21.15.138:0/3057415975 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:17.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.533+0000 7f4665800640 1 -- 172.21.15.138:0/3057415975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46600cb580 msgr2=0x7f4660149b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:17.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.533+0000 7f4665800640 1 --2- 172.21.15.138:0/3057415975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46600cb580 0x7f4660149b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:17.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.533+0000 7f4665800640 1 -- 172.21.15.138:0/3057415975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f466006c660 msgr2=0x7f466015abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:17.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.533+0000 7f4665800640 1 --2- 172.21.15.138:0/3057415975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f466006c660 0x7f466015abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:17.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.533+0000 7f4665800640 1 -- 172.21.15.138:0/3057415975 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4650009400 con 0x7f4660153f00 2024-01-21T22:25:17.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.535+0000 7f4665800640 1 --2- 172.21.15.138:0/3057415975 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4660153f00 0x7f466014a050 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f465800ca70 tx=0x7f465800cf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:17.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.539+0000 7f465cc00640 1 -- 172.21.15.138:0/3057415975 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465800abd0 con 0x7f4660153f00 2024-01-21T22:25:17.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.539+0000 7f465cc00640 1 -- 172.21.15.138:0/3057415975 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465800ad30 con 0x7f4660153f00 2024-01-21T22:25:17.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.539+0000 7f465cc00640 1 -- 172.21.15.138:0/3057415975 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465800aea0 con 0x7f4660153f00 2024-01-21T22:25:17.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.539+0000 7f4667800640 1 -- 172.21.15.138:0/3057415975 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f466015c620 con 0x7f4660153f00 2024-01-21T22:25:17.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.539+0000 7f4667800640 1 -- 172.21.15.138:0/3057415975 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f466014ede0 con 0x7f4660153f00 2024-01-21T22:25:17.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.540+0000 7f4667800640 1 -- 172.21.15.138:0/3057415975 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f466004eaa0 con 0x7f4660153f00 2024-01-21T22:25:17.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.564+0000 7f465cc00640 1 -- 172.21.15.138:0/3057415975 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f4658007070 con 0x7f4660153f00 2024-01-21T22:25:17.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.564+0000 7f465cc00640 1 --2- 172.21.15.138:0/3057415975 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f463806b380 0x7f463806d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:17.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.565+0000 7f4664e00640 1 --2- 172.21.15.138:0/3057415975 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f463806b380 0x7f463806d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:17.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.566+0000 7f4664e00640 1 --2- 172.21.15.138:0/3057415975 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f463806b380 0x7f463806d840 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4650036000 tx=0x7f4650030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:17.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.791+0000 7f465cc00640 1 -- 172.21.15.138:0/3057415975 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46580588a0 con 0x7f4660153f00 2024-01-21T22:25:17.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:17.846+0000 7f465cc00640 1 -- 172.21.15.138:0/3057415975 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f4658054be0 con 0x7f4660153f00 2024-01-21T22:25:18.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.043+0000 7f4667800640 1 -- 172.21.15.138:0/3057415975 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46600cac70 con 0x7f4660153f00 2024-01-21T22:25:18.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.087+0000 7f465cc00640 1 -- 172.21.15.138:0/3057415975 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16522 (secure 0 0 0) 0x7f4658017090 con 0x7f4660153f00 2024-01-21T22:25:18.089 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:25:18.090 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":23,"fsid":"d488d446-149f-48d0-89ca-b2830ae72846","created":"2024-01-21T22:24:28.581932+0000","modified":"2024-01-21T22:25:06.475790+0000","last_up_change":"2024-01-21T22:24:47.145799+0000","last_in_change":"2024-01-21T22:24:37.672807+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-21T22:24:49.586178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-21T22:25:02.044454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-21T22:25:03.361765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d812b2d6-c8a3-4015-bfdb-5766b25eae5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6825","nonce":2849203277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6827","nonce":2849203277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6831","nonce":2849203277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6829","nonce":2849203277}]},"public_addr":"172.21.15.138:6825/2849203277","cluster_addr":"172.21.15.138:6827/2849203277","heartbeat_back_addr":"172.21.15.138:6831/2849203277","heartbeat_front_addr":"172.21.15.138:6829/2849203277","state":["exists","up"]},{"osd":1,"uuid":"d8d875df-c840-46cc-b6c0-a847f9c93999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6817","nonce":174521731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6819","nonce":174521731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6823","nonce":174521731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6821","nonce":174521731}]},"public_addr":"172.21.15.138:6817/174521731","cluster_addr":"172.21.15.138:6819/174521731","heartbeat_back_addr":"172.21.15.138:6823/174521731","heartbeat_front_addr":"172.21.15.138:6821/174521731","state":["exists","up"]},{"osd":2,"uuid":"4bd94373-76ca-47a6-870b-16d96d7c9e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6801","nonce":1365838748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6803","nonce":1365838748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6809","nonce":1365838748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6805","nonce":1365838748}]},"public_addr":"172.21.15.138:6801/1365838748","cluster_addr":"172.21.15.138:6803/1365838748","heartbeat_back_addr":"172.21.15.138:6809/1365838748","heartbeat_front_addr":"172.21.15.138:6805/1365838748","state":["exists","up"]},{"osd":3,"uuid":"07267087-d4d2-4e88-ab9b-bb87f6dd061a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6808","nonce":1841873885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6811","nonce":1841873885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6815","nonce":1841873885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6813","nonce":1841873885}]},"public_addr":"172.21.15.138:6808/1841873885","cluster_addr":"172.21.15.138:6811/1841873885","heartbeat_back_addr":"172.21.15.138:6815/1841873885","heartbeat_front_addr":"172.21.15.138:6813/1841873885","state":["exists","up"]},{"osd":4,"uuid":"b47ef758-d783-4168-a9a5-11db3de4ca12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6827","nonce":3960812298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6829","nonce":3960812298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6833","nonce":3960812298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6831","nonce":3960812298}]},"public_addr":"172.21.15.152:6827/3960812298","cluster_addr":"172.21.15.152:6829/3960812298","heartbeat_back_addr":"172.21.15.152:6833/3960812298","heartbeat_front_addr":"172.21.15.152:6831/3960812298","state":["exists","up"]},{"osd":5,"uuid":"0ea5d43e-61d8-4823-815f-9e42c62fae97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6819","nonce":4095234314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6821","nonce":4095234314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6825","nonce":4095234314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6823","nonce":4095234314}]},"public_addr":"172.21.15.152:6819/4095234314","cluster_addr":"172.21.15.152:6821/4095234314","heartbeat_back_addr":"172.21.15.152:6825/4095234314","heartbeat_front_addr":"172.21.15.152:6823/4095234314","state":["exists","up"]},{"osd":6,"uuid":"59ffac36-e2e0-4693-ac82-ad5eccf6224d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6803","nonce":223848127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6805","nonce":223848127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6809","nonce":223848127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6807","nonce":223848127}]},"public_addr":"172.21.15.152:6803/223848127","cluster_addr":"172.21.15.152:6805/223848127","heartbeat_back_addr":"172.21.15.152:6809/223848127","heartbeat_front_addr":"172.21.15.152:6807/223848127","state":["exists","up"]},{"osd":7,"uuid":"4f892c7d-32a5-4960-bf88-4ed7c863cd0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6811","nonce":52509095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6813","nonce":52509095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6817","nonce":52509095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6815","nonce":52509095}]},"public_addr":"172.21.15.152:6811/52509095","cluster_addr":"172.21.15.152:6813/52509095","heartbeat_back_addr":"172.21.15.152:6817/52509095","heartbeat_front_addr":"172.21.15.152:6815/52509095","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.434460+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.374709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.613134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.298148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.242144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.198102+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.093385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.197180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-21T22:25:18.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.090+0000 7f4667800640 1 -- 172.21.15.138:0/3057415975 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f463806b380 msgr2=0x7f463806d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:18.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.090+0000 7f4667800640 1 --2- 172.21.15.138:0/3057415975 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f463806b380 0x7f463806d840 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4650036000 tx=0x7f4650030000 comp rx=0 tx=0).stop 2024-01-21T22:25:18.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.090+0000 7f4667800640 1 -- 172.21.15.138:0/3057415975 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4660153f00 msgr2=0x7f466014a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:18.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.090+0000 7f4667800640 1 --2- 172.21.15.138:0/3057415975 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4660153f00 0x7f466014a050 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f465800ca70 tx=0x7f465800cf30 comp rx=0 tx=0).stop 2024-01-21T22:25:18.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.090+0000 7f4667800640 1 -- 172.21.15.138:0/3057415975 shutdown_connections 2024-01-21T22:25:18.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.090+0000 7f4667800640 1 --2- 172.21.15.138:0/3057415975 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f463806b380 0x7f463806d840 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:18.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.090+0000 7f4667800640 1 --2- 172.21.15.138:0/3057415975 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4660153f00 0x7f466014a050 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:18.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.090+0000 7f4667800640 1 --2- 172.21.15.138:0/3057415975 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46600cb580 0x7f4660149b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:18.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.090+0000 7f4667800640 1 --2- 172.21.15.138:0/3057415975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f466006c660 0x7f466015abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:18.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.090+0000 7f4667800640 1 -- 172.21.15.138:0/3057415975 >> 172.21.15.138:0/3057415975 conn(0x7f4660066820 msgr2=0x7f46600cc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:18.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.090+0000 7f4667800640 1 -- 172.21.15.138:0/3057415975 shutdown_connections 2024-01-21T22:25:18.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.090+0000 7f4667800640 1 -- 172.21.15.138:0/3057415975 wait complete. 2024-01-21T22:25:18.111 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-01-21T22:25:18.112 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-21T22:25:18.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.318+0000 7f534f400640 1 Processor -- start 2024-01-21T22:25:18.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.318+0000 7f534f400640 1 -- start start 2024-01-21T22:25:18.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.320+0000 7f534f400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5348151e10 0x7f534815ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:18.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.320+0000 7f534f400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53481583d0 con 0x7f5348151a30 2024-01-21T22:25:18.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.320+0000 7f534f400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f534806c4d0 con 0x7f534815b3a0 2024-01-21T22:25:18.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.320+0000 7f534f400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f534806c7b0 con 0x7f5348151e10 2024-01-21T22:25:18.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.320+0000 7f5347e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5348151e10 0x7f534815ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:18.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.321+0000 7f534ca00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f5348151a30 0x7f53480c5ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:37868/0 (socket says 172.21.15.138:37868) 2024-01-21T22:25:18.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.321+0000 7f534ca00640 1 -- 172.21.15.138:0/1207029330 learned_addr learned my addr 172.21.15.138:0/1207029330 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:18.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.324+0000 7f5347400640 1 -- 172.21.15.138:0/1207029330 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 655147625 0 0) 0x7f53481583d0 con 0x7f5348151a30 2024-01-21T22:25:18.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.324+0000 7f5347400640 1 -- 172.21.15.138:0/1207029330 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5328003520 con 0x7f5348151a30 2024-01-21T22:25:18.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.324+0000 7f5347e00640 1 -- 172.21.15.138:0/1207029330 >> v1:172.21.15.152:6789/0 conn(0x7f534815b3a0 legacy=0x7f53480c47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:18.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.324+0000 7f5347e00640 1 -- 172.21.15.138:0/1207029330 >> v1:172.21.15.138:6789/0 conn(0x7f5348151a30 legacy=0x7f53480c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:18.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.324+0000 7f5347e00640 1 -- 172.21.15.138:0/1207029330 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5348157aa0 con 0x7f5348151e10 2024-01-21T22:25:18.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.329+0000 7f5347e00640 1 --2- 172.21.15.138:0/1207029330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5348151e10 0x7f534815ae60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f53300097a0 tx=0x7f533002b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae9badf22305de3f server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:18.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.333+0000 7f5347400640 1 -- 172.21.15.138:0/1207029330 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533002f810 con 0x7f5348151e10 2024-01-21T22:25:18.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.333+0000 7f5347400640 1 -- 172.21.15.138:0/1207029330 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533002f970 con 0x7f5348151e10 2024-01-21T22:25:18.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.333+0000 7f5347400640 1 -- 172.21.15.138:0/1207029330 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f533002fd00 con 0x7f5348151e10 2024-01-21T22:25:18.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.334+0000 7f534f400640 1 -- 172.21.15.138:0/1207029330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5348151e10 msgr2=0x7f534815ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:18.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.334+0000 7f534f400640 1 --2- 172.21.15.138:0/1207029330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5348151e10 0x7f534815ae60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f53300097a0 tx=0x7f533002b9f0 comp rx=0 tx=0).stop 2024-01-21T22:25:18.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.334+0000 7f534f400640 1 -- 172.21.15.138:0/1207029330 shutdown_connections 2024-01-21T22:25:18.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.334+0000 7f534f400640 1 --2- 172.21.15.138:0/1207029330 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5348151e10 0x7f534815ae60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:18.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.334+0000 7f534f400640 1 -- 172.21.15.138:0/1207029330 >> 172.21.15.138:0/1207029330 conn(0x7f53480cb0b0 msgr2=0x7f53480cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:18.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.334+0000 7f534f400640 1 -- 172.21.15.138:0/1207029330 shutdown_connections 2024-01-21T22:25:18.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.334+0000 7f534f400640 1 -- 172.21.15.138:0/1207029330 wait complete. 2024-01-21T22:25:18.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.337+0000 7f534f400640 1 Processor -- start 2024-01-21T22:25:18.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.337+0000 7f534f400640 1 -- start start 2024-01-21T22:25:18.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.338+0000 7f534f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5348151a30 0x7f534814f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:18.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.338+0000 7f534f400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f534815b3a0 0x7f534815a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:18.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.338+0000 7f534f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f534814dba0 0x7f534814e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:18.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.338+0000 7f534f400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53481e4740 con 0x7f5348151a30 2024-01-21T22:25:18.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.338+0000 7f534f400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534814e570 con 0x7f534814dba0 2024-01-21T22:25:18.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.338+0000 7f534f400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f534814f890 con 0x7f534815b3a0 2024-01-21T22:25:18.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.339+0000 7f534d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f534814dba0 0x7f534814e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:18.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.339+0000 7f5347e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f534815b3a0 0x7f534815a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:18.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.339+0000 7f534d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f534814dba0 0x7f534814e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:36052/0 (socket says 172.21.15.138:36052) 2024-01-21T22:25:18.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.339+0000 7f534d400640 1 -- 172.21.15.138:0/81389123 learned_addr learned my addr 172.21.15.138:0/81389123 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:18.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.342+0000 7f534ca00640 1 --2- 172.21.15.138:0/81389123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5348151a30 0x7f534814f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:18.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.344+0000 7f534d400640 1 -- 172.21.15.138:0/81389123 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f534815b3a0 msgr2=0x7f534815a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:18.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.344+0000 7f534d400640 1 --2- 172.21.15.138:0/81389123 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f534815b3a0 0x7f534815a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:18.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.344+0000 7f534d400640 1 -- 172.21.15.138:0/81389123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5348151a30 msgr2=0x7f534814f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:18.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.344+0000 7f534d400640 1 --2- 172.21.15.138:0/81389123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5348151a30 0x7f534814f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:18.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.344+0000 7f534d400640 1 -- 172.21.15.138:0/81389123 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5330009400 con 0x7f534814dba0 2024-01-21T22:25:18.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.348+0000 7f534d400640 1 --2- 172.21.15.138:0/81389123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f534814dba0 0x7f534814e030 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f534000c6e0 tx=0x7f534000cba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:18.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.353+0000 7f5344c00640 1 -- 172.21.15.138:0/81389123 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5340003b70 con 0x7f534814dba0 2024-01-21T22:25:18.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.353+0000 7f5344c00640 1 -- 172.21.15.138:0/81389123 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5340003cd0 con 0x7f534814dba0 2024-01-21T22:25:18.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.353+0000 7f5344c00640 1 -- 172.21.15.138:0/81389123 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5340005310 con 0x7f534814dba0 2024-01-21T22:25:18.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.353+0000 7f534f400640 1 -- 172.21.15.138:0/81389123 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53481568a0 con 0x7f534814dba0 2024-01-21T22:25:18.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.354+0000 7f534f400640 1 -- 172.21.15.138:0/81389123 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53481558a0 con 0x7f534814dba0 2024-01-21T22:25:18.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.355+0000 7f534f400640 1 -- 172.21.15.138:0/81389123 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f534806ccb0 con 0x7f534814dba0 2024-01-21T22:25:18.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.375+0000 7f5344c00640 1 -- 172.21.15.138:0/81389123 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f5340005470 con 0x7f534814dba0 2024-01-21T22:25:18.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.376+0000 7f5344c00640 1 --2- 172.21.15.138:0/81389123 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f532006b380 0x7f532006d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:18.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.377+0000 7f534ca00640 1 --2- 172.21.15.138:0/81389123 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f532006b380 0x7f532006d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:18.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.377+0000 7f5344c00640 1 -- 172.21.15.138:0/81389123 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53400593a0 con 0x7f534814dba0 2024-01-21T22:25:18.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.377+0000 7f534ca00640 1 --2- 172.21.15.138:0/81389123 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f532006b380 0x7f532006d840 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5338005880 tx=0x7f5338009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:18.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.665+0000 7f5344c00640 1 -- 172.21.15.138:0/81389123 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f5340054fe0 con 0x7f534814dba0 2024-01-21T22:25:18.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.867+0000 7f534f400640 1 -- 172.21.15.138:0/81389123 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f534804ea80 con 0x7f534814dba0 2024-01-21T22:25:18.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.903+0000 7f5344c00640 1 -- 172.21.15.138:0/81389123 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16522 (secure 0 0 0) 0x7f5340058600 con 0x7f534814dba0 2024-01-21T22:25:18.907 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:25:18.907 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":23,"fsid":"d488d446-149f-48d0-89ca-b2830ae72846","created":"2024-01-21T22:24:28.581932+0000","modified":"2024-01-21T22:25:06.475790+0000","last_up_change":"2024-01-21T22:24:47.145799+0000","last_in_change":"2024-01-21T22:24:37.672807+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-21T22:24:49.586178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-21T22:25:02.044454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-21T22:25:03.361765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d812b2d6-c8a3-4015-bfdb-5766b25eae5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6825","nonce":2849203277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6827","nonce":2849203277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6831","nonce":2849203277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6829","nonce":2849203277}]},"public_addr":"172.21.15.138:6825/2849203277","cluster_addr":"172.21.15.138:6827/2849203277","heartbeat_back_addr":"172.21.15.138:6831/2849203277","heartbeat_front_addr":"172.21.15.138:6829/2849203277","state":["exists","up"]},{"osd":1,"uuid":"d8d875df-c840-46cc-b6c0-a847f9c93999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6817","nonce":174521731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6819","nonce":174521731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6823","nonce":174521731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6821","nonce":174521731}]},"public_addr":"172.21.15.138:6817/174521731","cluster_addr":"172.21.15.138:6819/174521731","heartbeat_back_addr":"172.21.15.138:6823/174521731","heartbeat_front_addr":"172.21.15.138:6821/174521731","state":["exists","up"]},{"osd":2,"uuid":"4bd94373-76ca-47a6-870b-16d96d7c9e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6801","nonce":1365838748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6803","nonce":1365838748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6809","nonce":1365838748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6805","nonce":1365838748}]},"public_addr":"172.21.15.138:6801/1365838748","cluster_addr":"172.21.15.138:6803/1365838748","heartbeat_back_addr":"172.21.15.138:6809/1365838748","heartbeat_front_addr":"172.21.15.138:6805/1365838748","state":["exists","up"]},{"osd":3,"uuid":"07267087-d4d2-4e88-ab9b-bb87f6dd061a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6808","nonce":1841873885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6811","nonce":1841873885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6815","nonce":1841873885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6813","nonce":1841873885}]},"public_addr":"172.21.15.138:6808/1841873885","cluster_addr":"172.21.15.138:6811/1841873885","heartbeat_back_addr":"172.21.15.138:6815/1841873885","heartbeat_front_addr":"172.21.15.138:6813/1841873885","state":["exists","up"]},{"osd":4,"uuid":"b47ef758-d783-4168-a9a5-11db3de4ca12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6827","nonce":3960812298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6829","nonce":3960812298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6833","nonce":3960812298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6831","nonce":3960812298}]},"public_addr":"172.21.15.152:6827/3960812298","cluster_addr":"172.21.15.152:6829/3960812298","heartbeat_back_addr":"172.21.15.152:6833/3960812298","heartbeat_front_addr":"172.21.15.152:6831/3960812298","state":["exists","up"]},{"osd":5,"uuid":"0ea5d43e-61d8-4823-815f-9e42c62fae97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6819","nonce":4095234314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6821","nonce":4095234314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6825","nonce":4095234314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6823","nonce":4095234314}]},"public_addr":"172.21.15.152:6819/4095234314","cluster_addr":"172.21.15.152:6821/4095234314","heartbeat_back_addr":"172.21.15.152:6825/4095234314","heartbeat_front_addr":"172.21.15.152:6823/4095234314","state":["exists","up"]},{"osd":6,"uuid":"59ffac36-e2e0-4693-ac82-ad5eccf6224d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6803","nonce":223848127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6805","nonce":223848127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6809","nonce":223848127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6807","nonce":223848127}]},"public_addr":"172.21.15.152:6803/223848127","cluster_addr":"172.21.15.152:6805/223848127","heartbeat_back_addr":"172.21.15.152:6809/223848127","heartbeat_front_addr":"172.21.15.152:6807/223848127","state":["exists","up"]},{"osd":7,"uuid":"4f892c7d-32a5-4960-bf88-4ed7c863cd0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6811","nonce":52509095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6813","nonce":52509095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6817","nonce":52509095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6815","nonce":52509095}]},"public_addr":"172.21.15.152:6811/52509095","cluster_addr":"172.21.15.152:6813/52509095","heartbeat_back_addr":"172.21.15.152:6817/52509095","heartbeat_front_addr":"172.21.15.152:6815/52509095","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.434460+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.374709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.613134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.298148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.242144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.198102+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.093385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.197180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-21T22:25:18.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.907+0000 7f534f400640 1 -- 172.21.15.138:0/81389123 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f532006b380 msgr2=0x7f532006d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:18.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.907+0000 7f534f400640 1 --2- 172.21.15.138:0/81389123 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f532006b380 0x7f532006d840 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5338005880 tx=0x7f5338009040 comp rx=0 tx=0).stop 2024-01-21T22:25:18.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.907+0000 7f534f400640 1 -- 172.21.15.138:0/81389123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f534814dba0 msgr2=0x7f534814e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:18.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.907+0000 7f534f400640 1 --2- 172.21.15.138:0/81389123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f534814dba0 0x7f534814e030 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f534000c6e0 tx=0x7f534000cba0 comp rx=0 tx=0).stop 2024-01-21T22:25:18.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.907+0000 7f534f400640 1 -- 172.21.15.138:0/81389123 shutdown_connections 2024-01-21T22:25:18.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.907+0000 7f534f400640 1 --2- 172.21.15.138:0/81389123 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f532006b380 0x7f532006d840 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:18.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.907+0000 7f534f400640 1 --2- 172.21.15.138:0/81389123 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f534814dba0 0x7f534814e030 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:18.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.907+0000 7f534f400640 1 --2- 172.21.15.138:0/81389123 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f534815b3a0 0x7f534815a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:18.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.907+0000 7f534f400640 1 --2- 172.21.15.138:0/81389123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5348151a30 0x7f534814f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:18.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.907+0000 7f534f400640 1 -- 172.21.15.138:0/81389123 >> 172.21.15.138:0/81389123 conn(0x7f53480cb0b0 msgr2=0x7f53480cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:18.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.907+0000 7f534f400640 1 -- 172.21.15.138:0/81389123 shutdown_connections 2024-01-21T22:25:18.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:18.907+0000 7f534f400640 1 -- 172.21.15.138:0/81389123 wait complete. 2024-01-21T22:25:18.928 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-01-21T22:25:18.928 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-01-21T22:25:18.929 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-01-21T22:25:18.929 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-01-21T22:25:18.930 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-01-21T22:25:18.930 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-01-21T22:25:18.930 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-01-21T22:25:18.931 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-01-21T22:25:19.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.230+0000 7fbb0ca00640 1 Processor -- start 2024-01-21T22:25:19.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.230+0000 7fbb0ca00640 1 -- start start 2024-01-21T22:25:19.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.232+0000 7fbb0ca00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbb080c0b20 0x7fbb080bfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.232+0000 7fbb0ca00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb080c7090 con 0x7fbb0806e450 2024-01-21T22:25:19.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.233+0000 7fbb0ca00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb08159b10 con 0x7fbb080c7540 2024-01-21T22:25:19.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.233+0000 7fbb0ca00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb0806cd40 con 0x7fbb080c0b20 2024-01-21T22:25:19.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.233+0000 7fbb07200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbb080c0b20 0x7fbb080bfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.234+0000 7fbb06800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fbb080c7540 0x7fbb08152cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:53496/0 (socket says 172.21.15.138:53496) 2024-01-21T22:25:19.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.234+0000 7fbb07200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbb080c0b20 0x7fbb080bfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:36064/0 (socket says 172.21.15.138:36064) 2024-01-21T22:25:19.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.234+0000 7fbb07200640 1 -- 172.21.15.138:0/2991009814 learned_addr learned my addr 172.21.15.138:0/2991009814 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:19.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.236+0000 7fbb07200640 1 -- 172.21.15.138:0/2991009814 >> v1:172.21.15.152:6789/0 conn(0x7fbb080c7540 legacy=0x7fbb08152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:19.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.237+0000 7fbb07200640 1 -- 172.21.15.138:0/2991009814 >> v1:172.21.15.138:6789/0 conn(0x7fbb0806e450 legacy=0x7fbb080c0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:19.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.238+0000 7fbb07200640 1 -- 172.21.15.138:0/2991009814 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0806cf50 con 0x7fbb080c0b20 2024-01-21T22:25:19.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.242+0000 7fbb07200640 1 --2- 172.21.15.138:0/2991009814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb080c0b20 0x7fbb080bfed0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbb00009e50 tx=0x7fbb0002bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a320f7b706671c9 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.243+0000 7f360e800640 1 Processor -- start 2024-01-21T22:25:19.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.244+0000 7f360e800640 1 -- start start 2024-01-21T22:25:19.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.246+0000 7fbb05400640 1 -- 172.21.15.138:0/2991009814 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0002ca80 con 0x7fbb080c0b20 2024-01-21T22:25:19.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.246+0000 7fbb05400640 1 -- 172.21.15.138:0/2991009814 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0002cbe0 con 0x7fbb080c0b20 2024-01-21T22:25:19.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.246+0000 7fbb05400640 1 -- 172.21.15.138:0/2991009814 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb000343f0 con 0x7fbb080c0b20 2024-01-21T22:25:19.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.246+0000 7f360e800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f360815cdc0 0x7f36080c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.247+0000 7f360e800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f360815d1a0 0x7f36080c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.247+0000 7f360e800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36080c4900 con 0x7f36080ca7b0 2024-01-21T22:25:19.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.247+0000 7f360e800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36080c52d0 con 0x7f360815cdc0 2024-01-21T22:25:19.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.248+0000 7f360e800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f360815dc90 con 0x7f360815d1a0 2024-01-21T22:25:19.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.248+0000 7fc1b2c00640 1 Processor -- start 2024-01-21T22:25:19.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.249+0000 7f360de00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f360815d1a0 0x7f36080c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.249+0000 7f6f21000640 1 Processor -- start 2024-01-21T22:25:19.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.249+0000 7f360ca00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f36080ca7b0 0x7f360806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:37888/0 (socket says 172.21.15.138:37888) 2024-01-21T22:25:19.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.249+0000 7f360ca00640 1 -- 172.21.15.138:0/66875729 learned_addr learned my addr 172.21.15.138:0/66875729 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:19.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.249+0000 7fc1b2c00640 1 -- start start 2024-01-21T22:25:19.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.249+0000 7f6f21000640 1 -- start start 2024-01-21T22:25:19.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.250+0000 7fbb0ca00640 1 -- 172.21.15.138:0/2991009814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb080c0b20 msgr2=0x7fbb080bfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.250+0000 7fbb0ca00640 1 --2- 172.21.15.138:0/2991009814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb080c0b20 0x7fbb080bfed0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbb00009e50 tx=0x7fbb0002bee0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.250+0000 7f4b4fa00640 1 Processor -- start 2024-01-21T22:25:19.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.250+0000 7f4b4fa00640 1 -- start start 2024-01-21T22:25:19.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.250+0000 7fbb0ca00640 1 -- 172.21.15.138:0/2991009814 shutdown_connections 2024-01-21T22:25:19.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.250+0000 7fbb0ca00640 1 --2- 172.21.15.138:0/2991009814 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb080c0b20 0x7fbb080bfed0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.250+0000 7fbb0ca00640 1 -- 172.21.15.138:0/2991009814 >> 172.21.15.138:0/2991009814 conn(0x7fbb0806d930 msgr2=0x7fbb0806dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:19.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.251+0000 7f6f21000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6f1c0c4390 0x7f6f1c06b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.251+0000 7fc1b2c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fc1ac0ca380 0x7fc1ac15a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.251+0000 7f6f21000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f1c0ca1f0 con 0x7f6f1c153bd0 2024-01-21T22:25:19.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.251+0000 7f6f21000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f1c0cbb50 con 0x7f6f1c0c4870 2024-01-21T22:25:19.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.251+0000 7f6f21000640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c0b2530 con 0x7f6f1c0c4390 2024-01-21T22:25:19.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.251+0000 7fc1b2c00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc1ac0c9aa0 0x7fc1ac0c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.252+0000 7fc1b2c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1ac0c4590 con 0x7fc1ac0ca760 2024-01-21T22:25:19.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.252+0000 7f4b4fa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4b4806b9b0 0x7f4b48161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.252+0000 7fbb0ca00640 1 -- 172.21.15.138:0/2991009814 shutdown_connections 2024-01-21T22:25:19.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.252+0000 7fc1b2c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1ac15cf10 con 0x7fc1ac0ca380 2024-01-21T22:25:19.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.252+0000 7fc1b2c00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac15d1b0 con 0x7fc1ac0c9aa0 2024-01-21T22:25:19.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.252+0000 7f4b4fa00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b48162340 con 0x7f4b48158110 2024-01-21T22:25:19.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.252+0000 7f4b4fa00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b4806be80 con 0x7f4b480c5970 2024-01-21T22:25:19.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.252+0000 7f4b4fa00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b4815ec60 con 0x7f4b4806b9b0 2024-01-21T22:25:19.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.252+0000 7f6f13e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6f1c0c4390 0x7f6f1c06b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.252+0000 7f360de00640 1 -- 172.21.15.138:0/66875729 >> v2:172.21.15.138:3300/0 conn(0x7f360815cdc0 msgr2=0x7f36080c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:19.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.252+0000 7f360de00640 1 --2- 172.21.15.138:0/66875729 >> v2:172.21.15.138:3300/0 conn(0x7f360815cdc0 0x7f36080c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.252+0000 7f360de00640 1 -- 172.21.15.138:0/66875729 >> v1:172.21.15.138:6789/0 conn(0x7f36080ca7b0 legacy=0x7f360806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:19.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.252+0000 7f360de00640 1 -- 172.21.15.138:0/66875729 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360806b4b0 con 0x7f360815d1a0 2024-01-21T22:25:19.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.252+0000 7f6f13e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6f1c0c4390 0x7f6f1c06b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42290/0 (socket says 172.21.15.138:42290) 2024-01-21T22:25:19.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.252+0000 7f6f13e00640 1 -- 172.21.15.138:0/727842105 learned_addr learned my addr 172.21.15.138:0/727842105 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:19.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.252+0000 7fc1b2200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc1ac0c9aa0 0x7fc1ac0c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.252+0000 7fbb0ca00640 1 -- 172.21.15.138:0/2991009814 wait complete. 2024-01-21T22:25:19.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.253+0000 7f318c400640 1 Processor -- start 2024-01-21T22:25:19.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.253+0000 7f318c400640 1 -- start start 2024-01-21T22:25:19.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.253+0000 7fc1b2200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc1ac0c9aa0 0x7fc1ac0c4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42300/0 (socket says 172.21.15.138:42300) 2024-01-21T22:25:19.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.253+0000 7fc1b2200640 1 -- 172.21.15.138:0/1051461594 learned_addr learned my addr 172.21.15.138:0/1051461594 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:19.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.254+0000 7fbb0ca00640 1 Processor -- start 2024-01-21T22:25:19.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.254+0000 7fbb0ca00640 1 -- start start 2024-01-21T22:25:19.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.255+0000 7f318c400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f3184153d20 0x7f31840ca710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.255+0000 7f318c400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3184154100 con 0x7f31840cc940 2024-01-21T22:25:19.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.255+0000 7f318c400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31840c47f0 con 0x7f31840c43e0 2024-01-21T22:25:19.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.255+0000 7f318c400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31840ca500 con 0x7f3184153d20 2024-01-21T22:25:19.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.255+0000 7f04ab200640 1 Processor -- start 2024-01-21T22:25:19.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.256+0000 7f360de00640 1 --2- 172.21.15.138:0/66875729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f360815d1a0 0x7f36080c4d90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3608074a90 tx=0x7f360002bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1448d3d6a17455f6 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.256+0000 7f6f13e00640 1 -- 172.21.15.138:0/727842105 >> v1:172.21.15.152:6789/0 conn(0x7f6f1c0c4870 legacy=0x7f6f1c159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:19.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.256+0000 7f6f13e00640 1 -- 172.21.15.138:0/727842105 >> v1:172.21.15.138:6789/0 conn(0x7f6f1c153bd0 legacy=0x7f6f1c0cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:19.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.256+0000 7f6f13e00640 1 -- 172.21.15.138:0/727842105 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1c0cb540 con 0x7f6f1c0c4390 2024-01-21T22:25:19.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.256+0000 7fc1b1800640 1 --2- 172.21.15.138:0/1051461594 >> v2:172.21.15.152:3300/0 conn(0x7fc1ac0ca380 0x7fc1ac15a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.256+0000 7fbb0ca00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb0806e450 0x7fbb08157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.256+0000 7fbb0ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb080c0b20 0x7fbb0813afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.256+0000 7fbb0ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb080c7540 0x7fbb08162920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.256+0000 7fbb0ca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb081f3690 con 0x7fbb080c7540 2024-01-21T22:25:19.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.256+0000 7fbb0ca00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0813b770 con 0x7fbb080c0b20 2024-01-21T22:25:19.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.256+0000 7fbb0ca00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0813b4e0 con 0x7fbb0806e450 2024-01-21T22:25:19.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.256+0000 7f04ab200640 1 -- start start 2024-01-21T22:25:19.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.257+0000 7f04ab200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f04ac153b30 0x7f04ac0cc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.257+0000 7f04ab200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04ac0c9fa0 0x7f04ac15a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.258+0000 7f04ab200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f04ac0c40c0 0x7f04ac0cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.258+0000 7f4b4da00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f4b480c5970 0x7f4b480701d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:53516/0 (socket says 172.21.15.138:53516) 2024-01-21T22:25:19.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.258+0000 7f4b4da00640 1 -- 172.21.15.138:0/2021428752 learned_addr learned my addr 172.21.15.138:0/2021428752 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:19.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.258+0000 7f04ab200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04ac0ccb80 con 0x7f04ac153b30 2024-01-21T22:25:19.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.258+0000 7f04ab200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04ac0cb960 con 0x7f04ac0c40c0 2024-01-21T22:25:19.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.258+0000 7f04ab200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac0ca470 con 0x7f04ac0c9fa0 2024-01-21T22:25:19.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.258+0000 7fc1b2200640 1 -- 172.21.15.138:0/1051461594 >> v2:172.21.15.152:3300/0 conn(0x7fc1ac0ca380 msgr2=0x7fc1ac15a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:19.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.258+0000 7fc1b2200640 1 --2- 172.21.15.138:0/1051461594 >> v2:172.21.15.152:3300/0 conn(0x7fc1ac0ca380 0x7fc1ac15a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.258+0000 7fc1b2200640 1 -- 172.21.15.138:0/1051461594 >> v1:172.21.15.138:6789/0 conn(0x7fc1ac0ca760 legacy=0x7fc1ac1537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:19.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.258+0000 7fc1b2200640 1 -- 172.21.15.138:0/1051461594 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ac15d470 con 0x7fc1ac0c9aa0 2024-01-21T22:25:19.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.261+0000 7f6f13e00640 1 --2- 172.21.15.138:0/727842105 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6f1c0c4390 0x7f6f1c06b860 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6f040098a0 tx=0x7f6f0402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bdf43d7ae433dcf server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.261+0000 7fc1b2200640 1 --2- 172.21.15.138:0/1051461594 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc1ac0c9aa0 0x7fc1ac0c4a20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0098a0 tx=0x7fc19c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c81f1a27c36323ae server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.262+0000 7f4b47e00640 1 --2- 172.21.15.138:0/2021428752 >> v2:172.21.15.138:3300/0 conn(0x7f4b4806b9b0 0x7f4b48161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.262+0000 7f04a9400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04ac0c9fa0 0x7f04ac15a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.262+0000 7f318a400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f31840c43e0 0x7f31840cf260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:53522/0 (socket says 172.21.15.138:53522) 2024-01-21T22:25:19.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.262+0000 7f318a400640 1 -- 172.21.15.138:0/1340387101 learned_addr learned my addr 172.21.15.138:0/1340387101 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:19.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.263+0000 7f3189a00640 1 --2- 172.21.15.138:0/1340387101 >> v2:172.21.15.138:3300/0 conn(0x7f3184153d20 0x7f31840ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.263+0000 7f04a9400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04ac0c9fa0 0x7f04ac15a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42318/0 (socket says 172.21.15.138:42318) 2024-01-21T22:25:19.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.263+0000 7f04a9400640 1 -- 172.21.15.138:0/2066959456 learned_addr learned my addr 172.21.15.138:0/2066959456 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:19.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.263+0000 7fbe3f200640 1 Processor -- start 2024-01-21T22:25:19.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.264+0000 7fbe3f200640 1 -- start start 2024-01-21T22:25:19.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.265+0000 7fbe3f200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fbe380ca380 0x7fbe3815a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.265+0000 7fbe3f200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbe380ca760 0x7fbe381537e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.265+0000 7fbe3f200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe380c4590 con 0x7fbe380c9aa0 2024-01-21T22:25:19.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.265+0000 7fbe3f200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3815cf10 con 0x7fbe380ca380 2024-01-21T22:25:19.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.265+0000 7fbe3f200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3815d1b0 con 0x7fbe380ca760 2024-01-21T22:25:19.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.265+0000 7f4b47e00640 1 -- 172.21.15.138:0/2021428752 >> v1:172.21.15.152:6789/0 conn(0x7f4b480c5970 legacy=0x7f4b480701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.265+0000 7f4b47e00640 1 -- 172.21.15.138:0/2021428752 >> v1:172.21.15.138:6789/0 conn(0x7f4b48158110 legacy=0x7f4b4806c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:19.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.265+0000 7f4b47e00640 1 -- 172.21.15.138:0/2021428752 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4815dfe0 con 0x7f4b4806b9b0 2024-01-21T22:25:19.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.266+0000 7fbb06800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb0806e450 0x7fbb08157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.268+0000 7f3607e00640 1 -- 172.21.15.138:0/66875729 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360002cc80 con 0x7f360815d1a0 2024-01-21T22:25:19.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.268+0000 7f3607e00640 1 -- 172.21.15.138:0/66875729 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360002cde0 con 0x7f360815d1a0 2024-01-21T22:25:19.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.268+0000 7f3607e00640 1 -- 172.21.15.138:0/66875729 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3600034660 con 0x7f360815d1a0 2024-01-21T22:25:19.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.268+0000 7fbb05e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb080c0b20 0x7fbb0813afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.269+0000 7fbb06800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb0806e450 0x7fbb08157330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42302/0 (socket says 172.21.15.138:42302) 2024-01-21T22:25:19.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.269+0000 7fbb06800640 1 -- 172.21.15.138:0/1754551041 learned_addr learned my addr 172.21.15.138:0/1754551041 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:19.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.269+0000 7f4b47e00640 1 --2- 172.21.15.138:0/2021428752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b4806b9b0 0x7f4b48161e00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4b380097a0 tx=0x7f4b3802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586b433c726ac648 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.269+0000 7f360e800640 1 -- 172.21.15.138:0/66875729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f360815d1a0 msgr2=0x7f36080c4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.269+0000 7f360e800640 1 --2- 172.21.15.138:0/66875729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f360815d1a0 0x7f36080c4d90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3608074a90 tx=0x7f360002bed0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.269+0000 7f360e800640 1 -- 172.21.15.138:0/66875729 shutdown_connections 2024-01-21T22:25:19.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.269+0000 7f360e800640 1 --2- 172.21.15.138:0/66875729 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f360815d1a0 0x7f36080c4d90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.269+0000 7f360e800640 1 --2- 172.21.15.138:0/66875729 >> v2:172.21.15.138:3300/0 conn(0x7f360815cdc0 0x7f36080c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.269+0000 7f360e800640 1 -- 172.21.15.138:0/66875729 >> 172.21.15.138:0/66875729 conn(0x7f360806d370 msgr2=0x7f3608066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:19.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.269+0000 7f360e800640 1 -- 172.21.15.138:0/66875729 shutdown_connections 2024-01-21T22:25:19.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.269+0000 7f360e800640 1 -- 172.21.15.138:0/66875729 wait complete. 2024-01-21T22:25:19.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.271+0000 7f3189a00640 1 -- 172.21.15.138:0/1340387101 >> v1:172.21.15.152:6789/0 conn(0x7f31840c43e0 legacy=0x7f31840cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.271+0000 7f3189a00640 1 -- 172.21.15.138:0/1340387101 >> v1:172.21.15.138:6789/0 conn(0x7f31840cc940 legacy=0x7f318415a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:19.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.271+0000 7f04a9400640 1 -- 172.21.15.138:0/2066959456 >> v2:172.21.15.152:3300/0 conn(0x7f04ac0c40c0 msgr2=0x7f04ac0cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:19.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.271+0000 7f04a9400640 1 --2- 172.21.15.138:0/2066959456 >> v2:172.21.15.152:3300/0 conn(0x7f04ac0c40c0 0x7f04ac0cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.271+0000 7f04a9400640 1 -- 172.21.15.138:0/2066959456 >> v2:172.21.15.138:3300/0 conn(0x7f04ac153b30 msgr2=0x7f04ac0cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:19.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.271+0000 7f04a9400640 1 --2- 172.21.15.138:0/2066959456 >> v2:172.21.15.138:3300/0 conn(0x7f04ac153b30 0x7f04ac0cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.271+0000 7f04a9400640 1 -- 172.21.15.138:0/2066959456 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ac0cb3a0 con 0x7f04ac0c9fa0 2024-01-21T22:25:19.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.271+0000 7f360e800640 1 Processor -- start 2024-01-21T22:25:19.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.271+0000 7fbe3d400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbe380ca760 0x7fbe381537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.271+0000 7f3189a00640 1 -- 172.21.15.138:0/1340387101 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31840cb5c0 con 0x7f3184153d20 2024-01-21T22:25:19.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.271+0000 7f360e800640 1 -- start start 2024-01-21T22:25:19.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.272+0000 7fbe3d400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbe380ca760 0x7fbe381537e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42320/0 (socket says 172.21.15.138:42320) 2024-01-21T22:25:19.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.272+0000 7fbe3d400640 1 -- 172.21.15.138:0/2920448677 learned_addr learned my addr 172.21.15.138:0/2920448677 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:19.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.272+0000 7f360e800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36080ca7b0 0x7f36081425b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.272+0000 7f360e800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f360815cdc0 0x7f36080b3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.272+0000 7f360e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f360815d1a0 0x7f36080b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.272+0000 7f360e800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36081f77c0 con 0x7f360815d1a0 2024-01-21T22:25:19.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.272+0000 7f360e800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3608150ee0 con 0x7f360815cdc0 2024-01-21T22:25:19.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.272+0000 7f360e800640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3608142af0 con 0x7f36080ca7b0 2024-01-21T22:25:19.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.276+0000 7fbb06800640 1 -- 172.21.15.138:0/1754551041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb080c0b20 msgr2=0x7fbb0813afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.276+0000 7fbb06800640 1 --2- 172.21.15.138:0/1754551041 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb080c0b20 0x7fbb0813afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.276+0000 7fbb06800640 1 -- 172.21.15.138:0/1754551041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb080c7540 msgr2=0x7fbb08162920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.276+0000 7fbb06800640 1 --2- 172.21.15.138:0/1754551041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb080c7540 0x7fbb08162920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.276+0000 7fbb06800640 1 -- 172.21.15.138:0/1754551041 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb00009b30 con 0x7fbb0806e450 2024-01-21T22:25:19.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.277+0000 7fc1abe00640 1 -- 172.21.15.138:0/1051461594 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c030840 con 0x7fc1ac0c9aa0 2024-01-21T22:25:19.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.277+0000 7fc1abe00640 1 -- 172.21.15.138:0/1051461594 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19c0309a0 con 0x7fc1ac0c9aa0 2024-01-21T22:25:19.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.277+0000 7fc1abe00640 1 -- 172.21.15.138:0/1051461594 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c030d30 con 0x7fc1ac0c9aa0 2024-01-21T22:25:19.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.277+0000 7f3189a00640 1 --2- 172.21.15.138:0/1340387101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3184153d20 0x7f31840ca710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f316c0097a0 tx=0x7f316c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307427a971dfe87f server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.278+0000 7fc1b2c00640 1 -- 172.21.15.138:0/1051461594 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc1ac0c9aa0 msgr2=0x7fc1ac0c4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.278+0000 7fc1b2c00640 1 --2- 172.21.15.138:0/1051461594 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc1ac0c9aa0 0x7fc1ac0c4a20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0098a0 tx=0x7fc19c02b980 comp rx=0 tx=0).stop 2024-01-21T22:25:19.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.278+0000 7fc1b2c00640 1 -- 172.21.15.138:0/1051461594 shutdown_connections 2024-01-21T22:25:19.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.278+0000 7fc1b2c00640 1 --2- 172.21.15.138:0/1051461594 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc1ac0c9aa0 0x7fc1ac0c4a20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.278+0000 7fc1b2c00640 1 --2- 172.21.15.138:0/1051461594 >> v2:172.21.15.152:3300/0 conn(0x7fc1ac0ca380 0x7fc1ac15a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.278+0000 7fc1b2c00640 1 -- 172.21.15.138:0/1051461594 >> 172.21.15.138:0/1051461594 conn(0x7fc1ac06ce60 msgr2=0x7fc1ac06d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:19.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.278+0000 7fc1b2c00640 1 -- 172.21.15.138:0/1051461594 shutdown_connections 2024-01-21T22:25:19.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.278+0000 7fc1b2c00640 1 -- 172.21.15.138:0/1051461594 wait complete. 2024-01-21T22:25:19.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.279+0000 7f04a9400640 1 --2- 172.21.15.138:0/2066959456 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04ac0c9fa0 0x7f04ac15a270 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0494009870 tx=0x7f049402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=958eb83743880b7f server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.280+0000 7fc1b2c00640 1 Processor -- start 2024-01-21T22:25:19.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.280+0000 7fc1b2c00640 1 -- start start 2024-01-21T22:25:19.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.281+0000 7fc1b2c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1ac0c9aa0 0x7fc1ac14a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.281+0000 7fc1b2c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1ac0ca380 0x7fc1ac1f0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.281+0000 7fc1b2c00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc1ac0ca760 0x7fc1ac14b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.281+0000 7fc1b2c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac0cd020 con 0x7fc1ac0ca380 2024-01-21T22:25:19.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.281+0000 7fc1b2c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac0ce2a0 con 0x7fc1ac0c9aa0 2024-01-21T22:25:19.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.281+0000 7fc1b2c00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac1f0c40 con 0x7fc1ac0ca760 2024-01-21T22:25:19.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.281+0000 7fbe3d400640 1 -- 172.21.15.138:0/2920448677 >> v2:172.21.15.152:3300/0 conn(0x7fbe380ca380 msgr2=0x7fbe3815a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:19.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.281+0000 7fbe3d400640 1 --2- 172.21.15.138:0/2920448677 >> v2:172.21.15.152:3300/0 conn(0x7fbe380ca380 0x7fbe3815a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.281+0000 7fbe3d400640 1 -- 172.21.15.138:0/2920448677 >> v1:172.21.15.152:6789/0 conn(0x7fbe380c9aa0 legacy=0x7fbe380c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:19.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.281+0000 7fbe3d400640 1 -- 172.21.15.138:0/2920448677 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3815d470 con 0x7fbe380ca760 2024-01-21T22:25:19.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.285+0000 7f360ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f360815cdc0 0x7f36080b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.285+0000 7fbb06800640 1 --2- 172.21.15.138:0/1754551041 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb0806e450 0x7fbb08157330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000c480 tx=0x7fbaf000c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.286+0000 7f4b47400640 1 -- 172.21.15.138:0/2021428752 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b38030610 con 0x7f4b4806b9b0 2024-01-21T22:25:19.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.286+0000 7f4b47400640 1 -- 172.21.15.138:0/2021428752 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b38030770 con 0x7f4b4806b9b0 2024-01-21T22:25:19.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.286+0000 7f4b47400640 1 -- 172.21.15.138:0/2021428752 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b38030b00 con 0x7f4b4806b9b0 2024-01-21T22:25:19.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.286+0000 7f360ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f360815cdc0 0x7f36080b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:36128/0 (socket says 172.21.15.138:36128) 2024-01-21T22:25:19.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.286+0000 7f360ca00640 1 -- 172.21.15.138:0/4155817968 learned_addr learned my addr 172.21.15.138:0/4155817968 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:19.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.287+0000 7f4b4fa00640 1 -- 172.21.15.138:0/2021428752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b4806b9b0 msgr2=0x7f4b48161e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.287+0000 7f4b4fa00640 1 --2- 172.21.15.138:0/2021428752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b4806b9b0 0x7f4b48161e00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4b380097a0 tx=0x7f4b3802b960 comp rx=0 tx=0).stop 2024-01-21T22:25:19.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.287+0000 7fc1b1800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1ac0c9aa0 0x7fc1ac14a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.287+0000 7f4b4fa00640 1 -- 172.21.15.138:0/2021428752 shutdown_connections 2024-01-21T22:25:19.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.287+0000 7f4b4fa00640 1 --2- 172.21.15.138:0/2021428752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b4806b9b0 0x7f4b48161e00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.287+0000 7f4b4fa00640 1 -- 172.21.15.138:0/2021428752 >> 172.21.15.138:0/2021428752 conn(0x7f4b480cb070 msgr2=0x7f4b480cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:19.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.287+0000 7f4b4fa00640 1 -- 172.21.15.138:0/2021428752 shutdown_connections 2024-01-21T22:25:19.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.287+0000 7f4b4fa00640 1 -- 172.21.15.138:0/2021428752 wait complete. 2024-01-21T22:25:19.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.288+0000 7fc1b1800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1ac0c9aa0 0x7fc1ac14a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:36138/0 (socket says 172.21.15.138:36138) 2024-01-21T22:25:19.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.288+0000 7fc1b1800640 1 -- 172.21.15.138:0/4148076636 learned_addr learned my addr 172.21.15.138:0/4148076636 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:19.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.288+0000 7f3183e00640 1 -- 172.21.15.138:0/1340387101 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c030610 con 0x7f3184153d20 2024-01-21T22:25:19.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.288+0000 7f3183e00640 1 -- 172.21.15.138:0/1340387101 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316c030770 con 0x7f3184153d20 2024-01-21T22:25:19.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.288+0000 7f3183e00640 1 -- 172.21.15.138:0/1340387101 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316c030b00 con 0x7f3184153d20 2024-01-21T22:25:19.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.289+0000 7f318c400640 1 -- 172.21.15.138:0/1340387101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3184153d20 msgr2=0x7f31840ca710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.289+0000 7f318c400640 1 --2- 172.21.15.138:0/1340387101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3184153d20 0x7f31840ca710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f316c0097a0 tx=0x7f316c02b960 comp rx=0 tx=0).stop 2024-01-21T22:25:19.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.289+0000 7f318c400640 1 -- 172.21.15.138:0/1340387101 shutdown_connections 2024-01-21T22:25:19.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.289+0000 7f318c400640 1 --2- 172.21.15.138:0/1340387101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3184153d20 0x7f31840ca710 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.289+0000 7f318c400640 1 -- 172.21.15.138:0/1340387101 >> 172.21.15.138:0/1340387101 conn(0x7f318406d0d0 msgr2=0x7f318406af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:19.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.289+0000 7f318c400640 1 -- 172.21.15.138:0/1340387101 shutdown_connections 2024-01-21T22:25:19.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.289+0000 7f318c400640 1 -- 172.21.15.138:0/1340387101 wait complete. 2024-01-21T22:25:19.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.289+0000 7f4b4fa00640 1 Processor -- start 2024-01-21T22:25:19.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.289+0000 7f4b4fa00640 1 -- start start 2024-01-21T22:25:19.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.290+0000 7f4b4fa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4806b9b0 0x7f4b4815f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.290+0000 7f4b4fa00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b480c5970 0x7f4b481f4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.290+0000 7f4b4fa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b48158110 0x7f4b48156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.290+0000 7f4b4fa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b481f3f80 con 0x7f4b48158110 2024-01-21T22:25:19.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.290+0000 7f4b4fa00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b481f3950 con 0x7f4b4806b9b0 2024-01-21T22:25:19.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.290+0000 7f4b4fa00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b480ca910 con 0x7f4b480c5970 2024-01-21T22:25:19.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.291+0000 7f318c400640 1 Processor -- start 2024-01-21T22:25:19.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.291+0000 7fbe3d400640 1 --2- 172.21.15.138:0/2920448677 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbe380ca760 0x7fbe381537e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009870 tx=0x7fbe2002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7242f86ccc7d53f5 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.291+0000 7f318c400640 1 -- start start 2024-01-21T22:25:19.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.292+0000 7f6f19400640 1 -- 172.21.15.138:0/727842105 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f04030840 con 0x7f6f1c0c4390 2024-01-21T22:25:19.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.292+0000 7f6f19400640 1 -- 172.21.15.138:0/727842105 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f040309a0 con 0x7f6f1c0c4390 2024-01-21T22:25:19.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.292+0000 7f6f19400640 1 -- 172.21.15.138:0/727842105 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f04030d30 con 0x7f6f1c0c4390 2024-01-21T22:25:19.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.292+0000 7f360ca00640 1 -- 172.21.15.138:0/4155817968 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36080ca7b0 msgr2=0x7f36081425b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.292+0000 7f360ca00640 1 --2- 172.21.15.138:0/4155817968 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f36080ca7b0 0x7f36081425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.292+0000 7f360ca00640 1 -- 172.21.15.138:0/4155817968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f360815d1a0 msgr2=0x7f36080b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.292+0000 7f360ca00640 1 --2- 172.21.15.138:0/4155817968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f360815d1a0 0x7f36080b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.292+0000 7f360ca00640 1 -- 172.21.15.138:0/4155817968 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3600009a70 con 0x7f360815cdc0 2024-01-21T22:25:19.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.292+0000 7f318c400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f31840c43e0 0x7f31840cd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.292+0000 7f318c400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31840cc940 0x7f3184149b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.292+0000 7f318c400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3184153d20 0x7f318414a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.292+0000 7f318c400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31840b52c0 con 0x7f31840cc940 2024-01-21T22:25:19.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.292+0000 7f318c400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31841542c0 con 0x7f3184153d20 2024-01-21T22:25:19.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.292+0000 7f318c400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31840ce450 con 0x7f31840c43e0 2024-01-21T22:25:19.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.292+0000 7f6f21000640 1 -- 172.21.15.138:0/727842105 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6f1c0c4390 msgr2=0x7f6f1c06b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.292+0000 7f6f21000640 1 --2- 172.21.15.138:0/727842105 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6f1c0c4390 0x7f6f1c06b860 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6f040098a0 tx=0x7f6f0402b980 comp rx=0 tx=0).stop 2024-01-21T22:25:19.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.293+0000 7f6f21000640 1 -- 172.21.15.138:0/727842105 shutdown_connections 2024-01-21T22:25:19.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.293+0000 7f6f21000640 1 --2- 172.21.15.138:0/727842105 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6f1c0c4390 0x7f6f1c06b860 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.293+0000 7f6f21000640 1 -- 172.21.15.138:0/727842105 >> 172.21.15.138:0/727842105 conn(0x7f6f1c066640 msgr2=0x7f6f1c066a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:19.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.293+0000 7f6f21000640 1 -- 172.21.15.138:0/727842105 shutdown_connections 2024-01-21T22:25:19.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.293+0000 7f6f21000640 1 -- 172.21.15.138:0/727842105 wait complete. 2024-01-21T22:25:19.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.293+0000 7fc1b0e00640 1 --2- 172.21.15.138:0/4148076636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1ac0ca380 0x7fc1ac1f0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.294+0000 7f6f21000640 1 Processor -- start 2024-01-21T22:25:19.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.294+0000 7f6f21000640 1 -- start start 2024-01-21T22:25:19.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.295+0000 7f6f21000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6f1c0c4390 0x7f6f1c1f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.295+0000 7f6f21000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f1c0c4870 0x7f6f1c1f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.295+0000 7f6f21000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f1c153bd0 0x7f6f1c15bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.295+0000 7f6f21000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c1e5110 con 0x7f6f1c0c4870 2024-01-21T22:25:19.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.295+0000 7f6f21000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c15d7b0 con 0x7f6f1c153bd0 2024-01-21T22:25:19.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.295+0000 7f6f21000640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c1f4120 con 0x7f6f1c0c4390 2024-01-21T22:25:19.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.295+0000 7fc1b1800640 1 -- 172.21.15.138:0/4148076636 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc1ac0ca760 msgr2=0x7fc1ac14b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.295+0000 7fc1b1800640 1 --2- 172.21.15.138:0/4148076636 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fc1ac0ca760 0x7fc1ac14b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.295+0000 7fc1b1800640 1 -- 172.21.15.138:0/4148076636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1ac0ca380 msgr2=0x7fc1ac1f0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.295+0000 7fc1b1800640 1 --2- 172.21.15.138:0/4148076636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc1ac0ca380 0x7fc1ac1f0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.295+0000 7fc1b1800640 1 -- 172.21.15.138:0/4148076636 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c0094d0 con 0x7fc1ac0c9aa0 2024-01-21T22:25:19.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.298+0000 7f04a8a00640 1 -- 172.21.15.138:0/2066959456 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494031640 con 0x7f04ac0c9fa0 2024-01-21T22:25:19.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.298+0000 7f04a8a00640 1 -- 172.21.15.138:0/2066959456 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04940317a0 con 0x7f04ac0c9fa0 2024-01-21T22:25:19.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.298+0000 7f04a8a00640 1 -- 172.21.15.138:0/2066959456 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494031b30 con 0x7f04ac0c9fa0 2024-01-21T22:25:19.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.298+0000 7f360ca00640 1 --2- 172.21.15.138:0/4155817968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f360815cdc0 0x7f36080b3610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f35f000c430 tx=0x7f35f000c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.298+0000 7f4b47e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b480c5970 0x7f4b481f4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.298+0000 7f4b47e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b480c5970 0x7f4b481f4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42338/0 (socket says 172.21.15.138:42338) 2024-01-21T22:25:19.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.298+0000 7f4b47e00640 1 -- 172.21.15.138:0/2394839599 learned_addr learned my addr 172.21.15.138:0/2394839599 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:19.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.299+0000 7f04ab200640 1 -- 172.21.15.138:0/2066959456 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04ac0c9fa0 msgr2=0x7f04ac15a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.299+0000 7f04ab200640 1 --2- 172.21.15.138:0/2066959456 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04ac0c9fa0 0x7f04ac15a270 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0494009870 tx=0x7f049402b8f0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.299+0000 7f04ab200640 1 -- 172.21.15.138:0/2066959456 shutdown_connections 2024-01-21T22:25:19.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.299+0000 7f04ab200640 1 --2- 172.21.15.138:0/2066959456 >> v2:172.21.15.152:3300/0 conn(0x7f04ac0c40c0 0x7f04ac0cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.299+0000 7f04ab200640 1 --2- 172.21.15.138:0/2066959456 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04ac0c9fa0 0x7f04ac15a270 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.299+0000 7f04ab200640 1 --2- 172.21.15.138:0/2066959456 >> v2:172.21.15.138:3300/0 conn(0x7f04ac153b30 0x7f04ac0cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.299+0000 7f04ab200640 1 -- 172.21.15.138:0/2066959456 >> 172.21.15.138:0/2066959456 conn(0x7f04ac072420 msgr2=0x7f04ac06cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:19.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.299+0000 7f04ab200640 1 -- 172.21.15.138:0/2066959456 shutdown_connections 2024-01-21T22:25:19.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.299+0000 7f04ab200640 1 -- 172.21.15.138:0/2066959456 wait complete. 2024-01-21T22:25:19.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.299+0000 7fc1b1800640 1 --2- 172.21.15.138:0/4148076636 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1ac0c9aa0 0x7fc1ac14a780 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4007ef0 tx=0x7fc1a400e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.301+0000 7f04ab200640 1 Processor -- start 2024-01-21T22:25:19.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.301+0000 7f04ab200640 1 -- start start 2024-01-21T22:25:19.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.302+0000 7f3189000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31840cc940 0x7f3184149b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.302+0000 7fbe3ca00640 1 -- 172.21.15.138:0/2920448677 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe20030840 con 0x7fbe380ca760 2024-01-21T22:25:19.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.302+0000 7fbe3ca00640 1 -- 172.21.15.138:0/2920448677 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe200309a0 con 0x7fbe380ca760 2024-01-21T22:25:19.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.302+0000 7fbe3ca00640 1 -- 172.21.15.138:0/2920448677 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe20030d30 con 0x7fbe380ca760 2024-01-21T22:25:19.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.302+0000 7fbafea00640 1 -- 172.21.15.138:0/1754551041 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0007480 con 0x7fbb0806e450 2024-01-21T22:25:19.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.302+0000 7fbafea00640 1 -- 172.21.15.138:0/1754551041 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf00075e0 con 0x7fbb0806e450 2024-01-21T22:25:19.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.302+0000 7fbafea00640 1 -- 172.21.15.138:0/1754551041 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0005a50 con 0x7fbb0806e450 2024-01-21T22:25:19.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.302+0000 7fbb0ca00640 1 -- 172.21.15.138:0/1754551041 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb081f38a0 con 0x7fbb0806e450 2024-01-21T22:25:19.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.303+0000 7fbb0ca00640 1 -- 172.21.15.138:0/1754551041 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb081f3e20 con 0x7fbb0806e450 2024-01-21T22:25:19.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.303+0000 7f6f13e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f1c0c4870 0x7f6f1c1f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.303+0000 7f04ab200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04ac0c40c0 0x7f04ac1f3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.303+0000 7f04ab200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04ac0c9fa0 0x7f04ac1f0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.303+0000 7fbe3f200640 1 -- 172.21.15.138:0/2920448677 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbe380ca760 msgr2=0x7fbe381537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.303+0000 7fbe3f200640 1 --2- 172.21.15.138:0/2920448677 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbe380ca760 0x7fbe381537e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbe20009870 tx=0x7fbe2002b980 comp rx=0 tx=0).stop 2024-01-21T22:25:19.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.303+0000 7f04ab200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04ac153b30 0x7f04ac14ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.303+0000 7f04ab200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac14aa00 con 0x7f04ac0c40c0 2024-01-21T22:25:19.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.303+0000 7f04ab200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac0b39c0 con 0x7f04ac153b30 2024-01-21T22:25:19.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.303+0000 7f04ab200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac0b3f30 con 0x7f04ac0c9fa0 2024-01-21T22:25:19.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.303+0000 7fbe3f200640 1 -- 172.21.15.138:0/2920448677 shutdown_connections 2024-01-21T22:25:19.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.303+0000 7fbe3f200640 1 --2- 172.21.15.138:0/2920448677 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbe380ca760 0x7fbe381537e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.303+0000 7fbe3f200640 1 --2- 172.21.15.138:0/2920448677 >> v2:172.21.15.152:3300/0 conn(0x7fbe380ca380 0x7fbe3815a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.303+0000 7fbe3f200640 1 -- 172.21.15.138:0/2920448677 >> 172.21.15.138:0/2920448677 conn(0x7fbe3806ce60 msgr2=0x7fbe3806d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:19.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.303+0000 7f4b4da00640 1 --2- 172.21.15.138:0/2394839599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b48158110 0x7f4b48156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.303+0000 7fbe3f200640 1 -- 172.21.15.138:0/2920448677 shutdown_connections 2024-01-21T22:25:19.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.303+0000 7f4b4d000640 1 --2- 172.21.15.138:0/2394839599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4806b9b0 0x7f4b4815f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.304+0000 7fbb0ca00640 1 -- 172.21.15.138:0/1754551041 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbb08152790 con 0x7fbb0806e450 2024-01-21T22:25:19.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.304+0000 7f6f13e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f1c0c4870 0x7f6f1c1f3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52914/0 (socket says 172.21.15.138:52914) 2024-01-21T22:25:19.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.304+0000 7f6f13e00640 1 -- 172.21.15.138:0/1550492377 learned_addr learned my addr 172.21.15.138:0/1550492377 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:19.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.303+0000 7fbe3f200640 1 -- 172.21.15.138:0/2920448677 wait complete. 2024-01-21T22:25:19.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.304+0000 7f3189000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31840cc940 0x7f3184149b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52908/0 (socket says 172.21.15.138:52908) 2024-01-21T22:25:19.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.304+0000 7f3189000640 1 -- 172.21.15.138:0/691470637 learned_addr learned my addr 172.21.15.138:0/691470637 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:19.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.305+0000 7fbe3f200640 1 Processor -- start 2024-01-21T22:25:19.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.306+0000 7fbe3f200640 1 -- start start 2024-01-21T22:25:19.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.306+0000 7f4b47e00640 1 -- 172.21.15.138:0/2394839599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4806b9b0 msgr2=0x7f4b4815f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.306+0000 7f4b47e00640 1 --2- 172.21.15.138:0/2394839599 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4b4806b9b0 0x7f4b4815f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.306+0000 7f4b47e00640 1 -- 172.21.15.138:0/2394839599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b48158110 msgr2=0x7f4b48156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.306+0000 7f4b47e00640 1 --2- 172.21.15.138:0/2394839599 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b48158110 0x7f4b48156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.306+0000 7f4b47e00640 1 -- 172.21.15.138:0/2394839599 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b38009400 con 0x7f4b480c5970 2024-01-21T22:25:19.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.306+0000 7fc1a9600640 1 -- 172.21.15.138:0/4148076636 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4014620 con 0x7fc1ac0c9aa0 2024-01-21T22:25:19.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.306+0000 7fc1a9600640 1 -- 172.21.15.138:0/4148076636 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a4014780 con 0x7fc1ac0c9aa0 2024-01-21T22:25:19.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.306+0000 7fc1b2c00640 1 -- 172.21.15.138:0/4148076636 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1ac14ae50 con 0x7fc1ac0c9aa0 2024-01-21T22:25:19.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.306+0000 7fc1a9600640 1 -- 172.21.15.138:0/4148076636 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4014b30 con 0x7fc1ac0c9aa0 2024-01-21T22:25:19.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.306+0000 7fc1b2c00640 1 -- 172.21.15.138:0/4148076636 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ac0cd4e0 con 0x7fc1ac0c9aa0 2024-01-21T22:25:19.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.306+0000 7fbe3f200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe380c9aa0 0x7fbe381581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.306+0000 7f318a400640 1 --2- 172.21.15.138:0/691470637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3184153d20 0x7f318414a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.306+0000 7fbe3f200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbe380ca380 0x7fbe380b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.307+0000 7fbe3f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe380ca760 0x7fbe38155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.307+0000 7fbe3f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe38154410 con 0x7fbe380ca760 2024-01-21T22:25:19.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.307+0000 7fbe3f200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe38154890 con 0x7fbe380c9aa0 2024-01-21T22:25:19.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.307+0000 7fbe3f200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe38154b70 con 0x7fbe380ca380 2024-01-21T22:25:19.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.307+0000 7fc1b2c00640 1 -- 172.21.15.138:0/4148076636 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc1ac15dd10 con 0x7fc1ac0c9aa0 2024-01-21T22:25:19.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.308+0000 7f3605600640 1 -- 172.21.15.138:0/4155817968 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f0005ce0 con 0x7f360815cdc0 2024-01-21T22:25:19.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.308+0000 7f3605600640 1 -- 172.21.15.138:0/4155817968 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f0005e40 con 0x7f360815cdc0 2024-01-21T22:25:19.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.308+0000 7f3605600640 1 -- 172.21.15.138:0/4155817968 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f0007310 con 0x7f360815cdc0 2024-01-21T22:25:19.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.308+0000 7f360e800640 1 -- 172.21.15.138:0/4155817968 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3608153160 con 0x7f360815cdc0 2024-01-21T22:25:19.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.308+0000 7f360e800640 1 -- 172.21.15.138:0/4155817968 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36081f6e90 con 0x7f360815cdc0 2024-01-21T22:25:19.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.309+0000 7f6f1a800640 1 --2- 172.21.15.138:0/1550492377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f1c153bd0 0x7f6f1c15bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.309+0000 7f360e800640 1 -- 172.21.15.138:0/4155817968 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f360806ba30 con 0x7f360815cdc0 2024-01-21T22:25:19.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.311+0000 7f4b47e00640 1 --2- 172.21.15.138:0/2394839599 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b480c5970 0x7f4b481f4410 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4b380098c0 tx=0x7f4b38004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.312+0000 7f6f13e00640 1 -- 172.21.15.138:0/1550492377 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6f1c0c4390 msgr2=0x7f6f1c1f34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.312+0000 7f6f13e00640 1 --2- 172.21.15.138:0/1550492377 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6f1c0c4390 0x7f6f1c1f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.312+0000 7f6f13e00640 1 -- 172.21.15.138:0/1550492377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f1c153bd0 msgr2=0x7f6f1c15bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.312+0000 7f6f13e00640 1 --2- 172.21.15.138:0/1550492377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6f1c153bd0 0x7f6f1c15bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.312+0000 7f6f13e00640 1 -- 172.21.15.138:0/1550492377 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f040094d0 con 0x7f6f1c0c4870 2024-01-21T22:25:19.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.312+0000 7f318a400640 1 -- 172.21.15.138:0/691470637 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f31840c43e0 msgr2=0x7f31840cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.312+0000 7f318a400640 1 --2- 172.21.15.138:0/691470637 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f31840c43e0 0x7f31840cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.312+0000 7f318a400640 1 -- 172.21.15.138:0/691470637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31840cc940 msgr2=0x7f3184149b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.312+0000 7f318a400640 1 --2- 172.21.15.138:0/691470637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f31840cc940 0x7f3184149b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.312+0000 7f318a400640 1 -- 172.21.15.138:0/691470637 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316c009400 con 0x7f3184153d20 2024-01-21T22:25:19.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.313+0000 7f04aa800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04ac153b30 0x7f04ac14ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.315+0000 7f04aa800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04ac153b30 0x7f04ac14ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:36172/0 (socket says 172.21.15.138:36172) 2024-01-21T22:25:19.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.315+0000 7f04aa800640 1 -- 172.21.15.138:0/4159512619 learned_addr learned my addr 172.21.15.138:0/4159512619 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:19.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.323+0000 7f6f13e00640 1 --2- 172.21.15.138:0/1550492377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f1c0c4870 0x7f6f1c1f3a20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6f04000c00 tx=0x7f6f04004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.329+0000 7f6f12000640 1 -- 172.21.15.138:0/1550492377 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f04039b60 con 0x7f6f1c0c4870 2024-01-21T22:25:19.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.329+0000 7f6f12000640 1 -- 172.21.15.138:0/1550492377 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f04039cc0 con 0x7f6f1c0c4870 2024-01-21T22:25:19.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.329+0000 7f6f12000640 1 -- 172.21.15.138:0/1550492377 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f04043630 con 0x7f6f1c0c4870 2024-01-21T22:25:19.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.329+0000 7f6f21000640 1 -- 172.21.15.138:0/1550492377 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f1c0b4790 con 0x7f6f1c0c4870 2024-01-21T22:25:19.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.329+0000 7f6f21000640 1 -- 172.21.15.138:0/1550492377 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f1c1e4da0 con 0x7f6f1c0c4870 2024-01-21T22:25:19.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.329+0000 7fbe3e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe380ca760 0x7fbe38155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.330+0000 7f04a9e00640 1 --2- 172.21.15.138:0/4159512619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04ac0c40c0 0x7f04ac1f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.330+0000 7f6f21000640 1 -- 172.21.15.138:0/1550492377 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6f1c14e700 con 0x7f6f1c0c4870 2024-01-21T22:25:19.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.331+0000 7fbe3e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe380ca760 0x7fbe38155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52936/0 (socket says 172.21.15.138:52936) 2024-01-21T22:25:19.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.331+0000 7fbe3e800640 1 -- 172.21.15.138:0/2586780836 learned_addr learned my addr 172.21.15.138:0/2586780836 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:19.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.334+0000 7fbafea00640 1 -- 172.21.15.138:0/1754551041 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fbaf001b3b0 con 0x7fbb0806e450 2024-01-21T22:25:19.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.335+0000 7fbafea00640 1 --2- 172.21.15.138:0/1754551041 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fbae006b3b0 0x7fbae006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.336+0000 7fbb05e00640 1 --2- 172.21.15.138:0/1754551041 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fbae006b3b0 0x7fbae006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.336+0000 7fbb05e00640 1 --2- 172.21.15.138:0/1754551041 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fbae006b3b0 0x7fbae006d870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbb081632f0 tx=0x7fbaf8009060 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.340+0000 7fbafea00640 1 -- 172.21.15.138:0/1754551041 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbaf00582b0 con 0x7fbb0806e450 2024-01-21T22:25:19.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.340+0000 7fbafea00640 1 --2- 172.21.15.138:0/1754551041 >> [v2:172.21.15.138:6824/2849203277,v1:172.21.15.138:6825/2849203277] conn(0x7fbae0073550 0x7fbae0075970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.341+0000 7fbafea00640 1 -- 172.21.15.138:0/1754551041 --> [v2:172.21.15.138:6824/2849203277,v1:172.21.15.138:6825/2849203277] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbae0076020 con 0x7fbae0073550 2024-01-21T22:25:19.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.341+0000 7fbafea00640 1 -- 172.21.15.138:0/1754551041 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbaf00545f0 con 0x7fbb0806e450 2024-01-21T22:25:19.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.341+0000 7fbb07200640 1 --2- 172.21.15.138:0/1754551041 >> [v2:172.21.15.138:6824/2849203277,v1:172.21.15.138:6825/2849203277] conn(0x7fbae0073550 0x7fbae0075970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.341+0000 7fbb07200640 1 --2- 172.21.15.138:0/1754551041 >> [v2:172.21.15.138:6824/2849203277,v1:172.21.15.138:6825/2849203277] conn(0x7fbae0073550 0x7fbae0075970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.342+0000 7fbafea00640 1 -- 172.21.15.138:0/1754551041 <== osd.0 v2:172.21.15.138:6824/2849203277 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbae0076020 con 0x7fbae0073550 2024-01-21T22:25:19.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.347+0000 7f04a9400640 1 --2- 172.21.15.138:0/4159512619 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04ac0c9fa0 0x7f04ac1f0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.350+0000 7f3605600640 1 -- 172.21.15.138:0/4155817968 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f35f0007470 con 0x7f360815cdc0 2024-01-21T22:25:19.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.350+0000 7fbe3d400640 1 --2- 172.21.15.138:0/2586780836 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbe380ca380 0x7fbe380b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.350+0000 7f3605600640 1 --2- 172.21.15.138:0/4155817968 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f35e006b380 0x7f35e006d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.351+0000 7f360d400640 1 --2- 172.21.15.138:0/4155817968 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f35e006b380 0x7f35e006d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.351+0000 7f360d400640 1 --2- 172.21.15.138:0/4155817968 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f35e006b380 0x7f35e006d840 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009710 tx=0x7f35f8006c90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.352+0000 7f4b44c00640 1 -- 172.21.15.138:0/2394839599 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b3803e040 con 0x7f4b480c5970 2024-01-21T22:25:19.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.352+0000 7f4b44c00640 1 -- 172.21.15.138:0/2394839599 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b38039c70 con 0x7f4b480c5970 2024-01-21T22:25:19.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.353+0000 7f4b44c00640 1 -- 172.21.15.138:0/2394839599 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b380443f0 con 0x7f4b480c5970 2024-01-21T22:25:19.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.353+0000 7f4b4fa00640 1 -- 172.21.15.138:0/2394839599 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b480cabf0 con 0x7f4b480c5970 2024-01-21T22:25:19.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.353+0000 7f4b4fa00640 1 -- 172.21.15.138:0/2394839599 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b48157230 con 0x7f4b480c5970 2024-01-21T22:25:19.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.354+0000 7f4b4fa00640 1 -- 172.21.15.138:0/2394839599 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4b4815e880 con 0x7f4b480c5970 2024-01-21T22:25:19.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.358+0000 7f04a9400640 1 -- 172.21.15.138:0/4159512619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04ac153b30 msgr2=0x7f04ac14ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.358+0000 7f04a9400640 1 --2- 172.21.15.138:0/4159512619 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f04ac153b30 0x7f04ac14ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.358+0000 7f04a9400640 1 -- 172.21.15.138:0/4159512619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04ac0c40c0 msgr2=0x7f04ac1f3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.358+0000 7f04a9400640 1 --2- 172.21.15.138:0/4159512619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04ac0c40c0 0x7f04ac1f3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.358+0000 7f04a9400640 1 -- 172.21.15.138:0/4159512619 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04940094d0 con 0x7f04ac0c9fa0 2024-01-21T22:25:19.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.361+0000 7fbe3e800640 1 -- 172.21.15.138:0/2586780836 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbe380ca380 msgr2=0x7fbe380b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.361+0000 7fbe3e800640 1 --2- 172.21.15.138:0/2586780836 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbe380ca380 0x7fbe380b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.361+0000 7fbe3e800640 1 -- 172.21.15.138:0/2586780836 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe380c9aa0 msgr2=0x7fbe381581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.361+0000 7fbe3e800640 1 --2- 172.21.15.138:0/2586780836 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbe380c9aa0 0x7fbe381581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.361+0000 7fbe3e800640 1 -- 172.21.15.138:0/2586780836 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe200094d0 con 0x7fbe380ca760 2024-01-21T22:25:19.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.364+0000 7f04a9400640 1 --2- 172.21.15.138:0/4159512619 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04ac0c9fa0 0x7f04ac1f0690 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0494033000 tx=0x7f04940046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.365+0000 7fbb0ca00640 1 -- 172.21.15.138:0/1754551041 --> [v2:172.21.15.138:6824/2849203277,v1:172.21.15.138:6825/2849203277] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbb08068250 con 0x7fbae0073550 2024-01-21T22:25:19.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.366+0000 7fbafea00640 1 -- 172.21.15.138:0/1754551041 <== osd.0 v2:172.21.15.138:6824/2849203277 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbb08068250 con 0x7fbae0073550 2024-01-21T22:25:19.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.366+0000 7fc1a9600640 1 -- 172.21.15.138:0/4148076636 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fc1a4014c90 con 0x7fc1ac0c9aa0 2024-01-21T22:25:19.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.366+0000 7fbb0ca00640 1 -- 172.21.15.138:0/1754551041 >> [v2:172.21.15.138:6824/2849203277,v1:172.21.15.138:6825/2849203277] conn(0x7fbae0073550 msgr2=0x7fbae0075970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.366+0000 7fbb0ca00640 1 --2- 172.21.15.138:0/1754551041 >> [v2:172.21.15.138:6824/2849203277,v1:172.21.15.138:6825/2849203277] conn(0x7fbae0073550 0x7fbae0075970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.366+0000 7fbb0ca00640 1 -- 172.21.15.138:0/1754551041 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fbae006b3b0 msgr2=0x7fbae006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.366+0000 7fbb0ca00640 1 --2- 172.21.15.138:0/1754551041 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fbae006b3b0 0x7fbae006d870 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbb081632f0 tx=0x7fbaf8009060 comp rx=0 tx=0).stop 2024-01-21T22:25:19.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.366+0000 7fbb0ca00640 1 -- 172.21.15.138:0/1754551041 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb0806e450 msgr2=0x7fbb08157330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.366+0000 7fbb0ca00640 1 --2- 172.21.15.138:0/1754551041 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb0806e450 0x7fbb08157330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000c480 tx=0x7fbaf000c940 comp rx=0 tx=0).stop 2024-01-21T22:25:19.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.366+0000 7fbb07200640 1 -- 172.21.15.138:0/1754551041 reap_dead start 2024-01-21T22:25:19.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.367+0000 7fbb0ca00640 1 -- 172.21.15.138:0/1754551041 shutdown_connections 2024-01-21T22:25:19.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.367+0000 7fbb0ca00640 1 -- 172.21.15.138:0/1754551041 >> 172.21.15.138:0/1754551041 conn(0x7fbb0806d930 msgr2=0x7fbb08071440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:19.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.367+0000 7fbb0ca00640 1 -- 172.21.15.138:0/1754551041 shutdown_connections 2024-01-21T22:25:19.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.367+0000 7fbb0ca00640 1 -- 172.21.15.138:0/1754551041 wait complete. 2024-01-21T22:25:19.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.367+0000 7fc1a9600640 1 --2- 172.21.15.138:0/4148076636 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fc18406b3b0 0x7fc18406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.367+0000 7fc1b0e00640 1 --2- 172.21.15.138:0/4148076636 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fc18406b3b0 0x7fc18406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.368+0000 7fc1b0e00640 1 --2- 172.21.15.138:0/4148076636 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fc18406b3b0 0x7fc18406d870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac1ea5f0 tx=0x7fc194009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.368+0000 7fc1a9600640 1 -- 172.21.15.138:0/4148076636 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1a405a400 con 0x7fc1ac0c9aa0 2024-01-21T22:25:19.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.369+0000 7fc1a9600640 1 --2- 172.21.15.138:0/4148076636 >> [v2:172.21.15.138:6806/1841873885,v1:172.21.15.138:6808/1841873885] conn(0x7fc184073550 0x7fc184075970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.369+0000 7fc1a9600640 1 -- 172.21.15.138:0/4148076636 --> [v2:172.21.15.138:6806/1841873885,v1:172.21.15.138:6808/1841873885] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc184076020 con 0x7fc184073550 2024-01-21T22:25:19.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.369+0000 7fc1b2200640 1 --2- 172.21.15.138:0/4148076636 >> [v2:172.21.15.138:6806/1841873885,v1:172.21.15.138:6808/1841873885] conn(0x7fc184073550 0x7fc184075970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.370+0000 7fc1b2200640 1 --2- 172.21.15.138:0/4148076636 >> [v2:172.21.15.138:6806/1841873885,v1:172.21.15.138:6808/1841873885] conn(0x7fc184073550 0x7fc184075970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.371+0000 7fc1a9600640 1 -- 172.21.15.138:0/4148076636 <== osd.3 v2:172.21.15.138:6806/1841873885 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc184076020 con 0x7fc184073550 2024-01-21T22:25:19.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.372+0000 7f6f12000640 1 -- 172.21.15.138:0/1550492377 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f6f0404c3b0 con 0x7f6f1c0c4870 2024-01-21T22:25:19.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.372+0000 7f3605600640 1 -- 172.21.15.138:0/4155817968 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35f0007720 con 0x7f360815cdc0 2024-01-21T22:25:19.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.373+0000 7f3605600640 1 --2- 172.21.15.138:0/4155817968 >> [v2:172.21.15.138:6816/174521731,v1:172.21.15.138:6817/174521731] conn(0x7f35e0073520 0x7f35e0075940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.373+0000 7f3605600640 1 -- 172.21.15.138:0/4155817968 --> [v2:172.21.15.138:6816/174521731,v1:172.21.15.138:6817/174521731] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35e0075ff0 con 0x7f35e0073520 2024-01-21T22:25:19.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.373+0000 7f3605600640 1 -- 172.21.15.138:0/4155817968 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35f0054670 con 0x7f360815cdc0 2024-01-21T22:25:19.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.373+0000 7fc1a9600640 1 -- 172.21.15.138:0/4148076636 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1a4056040 con 0x7fc1ac0c9aa0 2024-01-21T22:25:19.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.373+0000 7f360de00640 1 --2- 172.21.15.138:0/4155817968 >> [v2:172.21.15.138:6816/174521731,v1:172.21.15.138:6817/174521731] conn(0x7f35e0073520 0x7f35e0075940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.373+0000 7f6f12000640 1 --2- 172.21.15.138:0/1550492377 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6ef406b380 0x7f6ef406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.374+0000 7f360de00640 1 --2- 172.21.15.138:0/4155817968 >> [v2:172.21.15.138:6816/174521731,v1:172.21.15.138:6817/174521731] conn(0x7f35e0073520 0x7f35e0075940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.374+0000 7f6f19e00640 1 --2- 172.21.15.138:0/1550492377 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6ef406b380 0x7f6ef406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.374+0000 7f6f19e00640 1 --2- 172.21.15.138:0/1550492377 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6ef406b380 0x7f6ef406d840 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c001560 tx=0x7f6f0c007990 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.375+0000 7f3605600640 1 -- 172.21.15.138:0/4155817968 <== osd.1 v2:172.21.15.138:6816/174521731 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f35e0075ff0 con 0x7f35e0073520 2024-01-21T22:25:19.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.376+0000 7f318a400640 1 --2- 172.21.15.138:0/691470637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3184153d20 0x7f318414a0a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f317c00ca70 tx=0x7f317c00cf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.384+0000 7f6f12000640 1 -- 172.21.15.138:0/1550492377 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f04039e30 con 0x7f6f1c0c4870 2024-01-21T22:25:19.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.385+0000 7f6f12000640 1 --2- 172.21.15.138:0/1550492377 >> [v2:172.21.15.152:6826/3960812298,v1:172.21.15.152:6827/3960812298] conn(0x7f6ef4073520 0x7f6ef4075940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.385+0000 7f6f12000640 1 -- 172.21.15.138:0/1550492377 --> [v2:172.21.15.152:6826/3960812298,v1:172.21.15.152:6827/3960812298] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ef4075ff0 con 0x7f6ef4073520 2024-01-21T22:25:19.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.385+0000 7f6f12000640 1 -- 172.21.15.138:0/1550492377 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6f040776c0 con 0x7f6f1c0c4870 2024-01-21T22:25:19.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.385+0000 7fbe3e800640 1 --2- 172.21.15.138:0/2586780836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe380ca760 0x7fbe38155930 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000c780 tx=0x7fbe3000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.385+0000 7f6f1a800640 1 --2- 172.21.15.138:0/1550492377 >> [v2:172.21.15.152:6826/3960812298,v1:172.21.15.152:6827/3960812298] conn(0x7f6ef4073520 0x7f6ef4075940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.388 INFO:teuthology.orchestra.run.smithi138.stdout:60129542156 2024-01-21T22:25:19.389 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-21T22:25:19.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.386+0000 7f6f1a800640 1 --2- 172.21.15.138:0/1550492377 >> [v2:172.21.15.152:6826/3960812298,v1:172.21.15.152:6827/3960812298] conn(0x7f6ef4073520 0x7f6ef4075940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.386+0000 7f6f12000640 1 -- 172.21.15.138:0/1550492377 <== osd.4 v2:172.21.15.152:6826/3960812298 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6ef4075ff0 con 0x7f6ef4073520 2024-01-21T22:25:19.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.389+0000 7fc1b2c00640 1 -- 172.21.15.138:0/4148076636 --> [v2:172.21.15.138:6806/1841873885,v1:172.21.15.138:6808/1841873885] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc1ac14b9d0 con 0x7fc184073550 2024-01-21T22:25:19.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.389+0000 7fc1a9600640 1 -- 172.21.15.138:0/4148076636 <== osd.3 v2:172.21.15.138:6806/1841873885 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc1ac14b9d0 con 0x7fc184073550 2024-01-21T22:25:19.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.390+0000 7fc1b2c00640 1 -- 172.21.15.138:0/4148076636 >> [v2:172.21.15.138:6806/1841873885,v1:172.21.15.138:6808/1841873885] conn(0x7fc184073550 msgr2=0x7fc184075970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.390+0000 7fc1b2c00640 1 --2- 172.21.15.138:0/4148076636 >> [v2:172.21.15.138:6806/1841873885,v1:172.21.15.138:6808/1841873885] conn(0x7fc184073550 0x7fc184075970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.390+0000 7fc1b2c00640 1 -- 172.21.15.138:0/4148076636 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fc18406b3b0 msgr2=0x7fc18406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.390+0000 7fc1b2c00640 1 --2- 172.21.15.138:0/4148076636 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fc18406b3b0 0x7fc18406d870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac1ea5f0 tx=0x7fc194009040 comp rx=0 tx=0).stop 2024-01-21T22:25:19.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.390+0000 7fc1b2c00640 1 -- 172.21.15.138:0/4148076636 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1ac0c9aa0 msgr2=0x7fc1ac14a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.390+0000 7fc1b2c00640 1 --2- 172.21.15.138:0/4148076636 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fc1ac0c9aa0 0x7fc1ac14a780 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4007ef0 tx=0x7fc1a400e4b0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.390+0000 7fc1b2200640 1 -- 172.21.15.138:0/4148076636 reap_dead start 2024-01-21T22:25:19.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.390+0000 7fc1b2c00640 1 -- 172.21.15.138:0/4148076636 shutdown_connections 2024-01-21T22:25:19.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.390+0000 7fc1b2c00640 1 -- 172.21.15.138:0/4148076636 >> 172.21.15.138:0/4148076636 conn(0x7fc1ac06ce60 msgr2=0x7fc1ac157700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:19.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.390+0000 7fc1b2c00640 1 -- 172.21.15.138:0/4148076636 shutdown_connections 2024-01-21T22:25:19.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.390+0000 7fc1b2c00640 1 -- 172.21.15.138:0/4148076636 wait complete. 2024-01-21T22:25:19.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.390+0000 7f4b44c00640 1 -- 172.21.15.138:0/2394839599 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f4b3804d3b0 con 0x7f4b480c5970 2024-01-21T22:25:19.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.391+0000 7f4b44c00640 1 --2- 172.21.15.138:0/2394839599 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f4b2006b380 0x7f4b2006d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.391+0000 7f4b4d000640 1 --2- 172.21.15.138:0/2394839599 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f4b2006b380 0x7f4b2006d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.392+0000 7f4b44c00640 1 -- 172.21.15.138:0/2394839599 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b3807b1b0 con 0x7f4b480c5970 2024-01-21T22:25:19.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.392+0000 7f4b4d000640 1 --2- 172.21.15.138:0/2394839599 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f4b2006b380 0x7f4b2006d840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000a730 tx=0x7f4b30008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.393+0000 7f4b44c00640 1 --2- 172.21.15.138:0/2394839599 >> [v2:172.21.15.152:6818/4095234314,v1:172.21.15.152:6819/4095234314] conn(0x7f4b20073520 0x7f4b20075940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.393+0000 7f4b44c00640 1 -- 172.21.15.138:0/2394839599 --> [v2:172.21.15.152:6818/4095234314,v1:172.21.15.152:6819/4095234314] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b20075ff0 con 0x7f4b20073520 2024-01-21T22:25:19.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.393+0000 7f4b44c00640 1 -- 172.21.15.138:0/2394839599 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b380774f0 con 0x7f4b480c5970 2024-01-21T22:25:19.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.393+0000 7f4b4da00640 1 --2- 172.21.15.138:0/2394839599 >> [v2:172.21.15.152:6818/4095234314,v1:172.21.15.152:6819/4095234314] conn(0x7f4b20073520 0x7f4b20075940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.393+0000 7f4b4da00640 1 --2- 172.21.15.138:0/2394839599 >> [v2:172.21.15.152:6818/4095234314,v1:172.21.15.152:6819/4095234314] conn(0x7f4b20073520 0x7f4b20075940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.394+0000 7f360e800640 1 -- 172.21.15.138:0/4155817968 --> [v2:172.21.15.138:6816/174521731,v1:172.21.15.138:6817/174521731] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f36080c4010 con 0x7f35e0073520 2024-01-21T22:25:19.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.394+0000 7f4b44c00640 1 -- 172.21.15.138:0/2394839599 <== osd.5 v2:172.21.15.152:6818/4095234314 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4b20075ff0 con 0x7f4b20073520 2024-01-21T22:25:19.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.395+0000 7f3605600640 1 -- 172.21.15.138:0/4155817968 <== osd.1 v2:172.21.15.138:6816/174521731 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3600009450 con 0x7f35e0073520 2024-01-21T22:25:19.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.395+0000 7f360e800640 1 -- 172.21.15.138:0/4155817968 >> [v2:172.21.15.138:6816/174521731,v1:172.21.15.138:6817/174521731] conn(0x7f35e0073520 msgr2=0x7f35e0075940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.395+0000 7f360e800640 1 --2- 172.21.15.138:0/4155817968 >> [v2:172.21.15.138:6816/174521731,v1:172.21.15.138:6817/174521731] conn(0x7f35e0073520 0x7f35e0075940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.396+0000 7f360e800640 1 -- 172.21.15.138:0/4155817968 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f35e006b380 msgr2=0x7f35e006d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.396+0000 7f360e800640 1 --2- 172.21.15.138:0/4155817968 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f35e006b380 0x7f35e006d840 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009710 tx=0x7f35f8006c90 comp rx=0 tx=0).stop 2024-01-21T22:25:19.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.396+0000 7f360e800640 1 -- 172.21.15.138:0/4155817968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f360815cdc0 msgr2=0x7f36080b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.396+0000 7f360e800640 1 --2- 172.21.15.138:0/4155817968 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f360815cdc0 0x7f36080b3610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f35f000c430 tx=0x7f35f000c8f0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.396+0000 7f360de00640 1 -- 172.21.15.138:0/4155817968 reap_dead start 2024-01-21T22:25:19.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.396+0000 7f360e800640 1 -- 172.21.15.138:0/4155817968 shutdown_connections 2024-01-21T22:25:19.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.396+0000 7f360e800640 1 -- 172.21.15.138:0/4155817968 >> 172.21.15.138:0/4155817968 conn(0x7f360806d370 msgr2=0x7f360815a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:19.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.397+0000 7f360e800640 1 -- 172.21.15.138:0/4155817968 shutdown_connections 2024-01-21T22:25:19.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.397+0000 7f360e800640 1 -- 172.21.15.138:0/4155817968 wait complete. 2024-01-21T22:25:19.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.398+0000 7f3181600640 1 -- 172.21.15.138:0/691470637 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c00ab70 con 0x7f3184153d20 2024-01-21T22:25:19.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.398+0000 7f3181600640 1 -- 172.21.15.138:0/691470637 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317c00acd0 con 0x7f3184153d20 2024-01-21T22:25:19.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.398+0000 7f3181600640 1 -- 172.21.15.138:0/691470637 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c012620 con 0x7f3184153d20 2024-01-21T22:25:19.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.399+0000 7f318c400640 1 -- 172.21.15.138:0/691470637 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3184154580 con 0x7f3184153d20 2024-01-21T22:25:19.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.399+0000 7f318c400640 1 -- 172.21.15.138:0/691470637 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3184154b40 con 0x7f3184153d20 2024-01-21T22:25:19.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.400+0000 7f318c400640 1 -- 172.21.15.138:0/691470637 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f31840cbe60 con 0x7f3184153d20 2024-01-21T22:25:19.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.401+0000 7f04a2000640 1 -- 172.21.15.138:0/4159512619 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f049403ac60 con 0x7f04ac0c9fa0 2024-01-21T22:25:19.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.401+0000 7f04a2000640 1 -- 172.21.15.138:0/4159512619 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f049403adc0 con 0x7f04ac0c9fa0 2024-01-21T22:25:19.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.401+0000 7f04a2000640 1 -- 172.21.15.138:0/4159512619 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0494044630 con 0x7f04ac0c9fa0 2024-01-21T22:25:19.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.401+0000 7f04ab200640 1 -- 172.21.15.138:0/4159512619 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04ac0b4210 con 0x7f04ac0c9fa0 2024-01-21T22:25:19.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.402+0000 7f04ab200640 1 -- 172.21.15.138:0/4159512619 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04ac1f2830 con 0x7f04ac0c9fa0 2024-01-21T22:25:19.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.403+0000 7f04ab200640 1 -- 172.21.15.138:0/4159512619 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f04ac159d50 con 0x7f04ac0c9fa0 2024-01-21T22:25:19.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.402+0000 7fbe36000640 1 -- 172.21.15.138:0/2586780836 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30004b60 con 0x7fbe380ca760 2024-01-21T22:25:19.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.402+0000 7fbe36000640 1 -- 172.21.15.138:0/2586780836 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3000ce90 con 0x7fbe380ca760 2024-01-21T22:25:19.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.404+0000 7fbe3f200640 1 -- 172.21.15.138:0/2586780836 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe38154e50 con 0x7fbe380ca760 2024-01-21T22:25:19.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.404+0000 7fbe36000640 1 -- 172.21.15.138:0/2586780836 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3000a850 con 0x7fbe380ca760 2024-01-21T22:25:19.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.404+0000 7fbe3f200640 1 -- 172.21.15.138:0/2586780836 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe38155410 con 0x7fbe380ca760 2024-01-21T22:25:19.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.406+0000 7fbe3f200640 1 -- 172.21.15.138:0/2586780836 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbe3815cda0 con 0x7fbe380ca760 2024-01-21T22:25:19.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.411+0000 7f6f21000640 1 -- 172.21.15.138:0/1550492377 --> [v2:172.21.15.152:6826/3960812298,v1:172.21.15.152:6827/3960812298] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6f1c14e420 con 0x7f6ef4073520 2024-01-21T22:25:19.414 INFO:teuthology.orchestra.run.smithi138.stdout:60129542156 2024-01-21T22:25:19.414 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-21T22:25:19.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.412+0000 7f6f12000640 1 -- 172.21.15.138:0/1550492377 <== osd.4 v2:172.21.15.152:6826/3960812298 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6f1c14e420 con 0x7f6ef4073520 2024-01-21T22:25:19.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.412+0000 7f6f21000640 1 -- 172.21.15.138:0/1550492377 >> [v2:172.21.15.152:6826/3960812298,v1:172.21.15.152:6827/3960812298] conn(0x7f6ef4073520 msgr2=0x7f6ef4075940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.412+0000 7f6f21000640 1 --2- 172.21.15.138:0/1550492377 >> [v2:172.21.15.152:6826/3960812298,v1:172.21.15.152:6827/3960812298] conn(0x7f6ef4073520 0x7f6ef4075940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.413+0000 7f6f21000640 1 -- 172.21.15.138:0/1550492377 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6ef406b380 msgr2=0x7f6ef406d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.413+0000 7f6f21000640 1 --2- 172.21.15.138:0/1550492377 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6ef406b380 0x7f6ef406d840 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c001560 tx=0x7f6f0c007990 comp rx=0 tx=0).stop 2024-01-21T22:25:19.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.413+0000 7f6f21000640 1 -- 172.21.15.138:0/1550492377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f1c0c4870 msgr2=0x7f6f1c1f3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.413+0000 7f6f21000640 1 --2- 172.21.15.138:0/1550492377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6f1c0c4870 0x7f6f1c1f3a20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6f04000c00 tx=0x7f6f04004730 comp rx=0 tx=0).stop 2024-01-21T22:25:19.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.413+0000 7f6f1a800640 1 -- 172.21.15.138:0/1550492377 reap_dead start 2024-01-21T22:25:19.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.413+0000 7f6f21000640 1 -- 172.21.15.138:0/1550492377 shutdown_connections 2024-01-21T22:25:19.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.413+0000 7f6f21000640 1 -- 172.21.15.138:0/1550492377 >> 172.21.15.138:0/1550492377 conn(0x7f6f1c066640 msgr2=0x7f6f1c066a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:19.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.413+0000 7f6f21000640 1 -- 172.21.15.138:0/1550492377 shutdown_connections 2024-01-21T22:25:19.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.413+0000 7f6f21000640 1 -- 172.21.15.138:0/1550492377 wait complete. 2024-01-21T22:25:19.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.419+0000 7f4b4fa00640 1 -- 172.21.15.138:0/2394839599 --> [v2:172.21.15.152:6818/4095234314,v1:172.21.15.152:6819/4095234314] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4b4815e5a0 con 0x7f4b20073520 2024-01-21T22:25:19.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.419+0000 7f4b44c00640 1 -- 172.21.15.138:0/2394839599 <== osd.5 v2:172.21.15.152:6818/4095234314 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4b40001f00 con 0x7f4b20073520 2024-01-21T22:25:19.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.420+0000 7f4b4fa00640 1 -- 172.21.15.138:0/2394839599 >> [v2:172.21.15.152:6818/4095234314,v1:172.21.15.152:6819/4095234314] conn(0x7f4b20073520 msgr2=0x7f4b20075940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.420+0000 7f4b4fa00640 1 --2- 172.21.15.138:0/2394839599 >> [v2:172.21.15.152:6818/4095234314,v1:172.21.15.152:6819/4095234314] conn(0x7f4b20073520 0x7f4b20075940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.420+0000 7f4b4fa00640 1 -- 172.21.15.138:0/2394839599 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f4b2006b380 msgr2=0x7f4b2006d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.420+0000 7f4b4fa00640 1 --2- 172.21.15.138:0/2394839599 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f4b2006b380 0x7f4b2006d840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4b3000a730 tx=0x7f4b30008040 comp rx=0 tx=0).stop 2024-01-21T22:25:19.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.420+0000 7f4b4fa00640 1 -- 172.21.15.138:0/2394839599 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b480c5970 msgr2=0x7f4b481f4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.420+0000 7f4b4fa00640 1 --2- 172.21.15.138:0/2394839599 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4b480c5970 0x7f4b481f4410 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4b380098c0 tx=0x7f4b38004730 comp rx=0 tx=0).stop 2024-01-21T22:25:19.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.420+0000 7f4b4da00640 1 -- 172.21.15.138:0/2394839599 reap_dead start 2024-01-21T22:25:19.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.420+0000 7f4b4fa00640 1 -- 172.21.15.138:0/2394839599 shutdown_connections 2024-01-21T22:25:19.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.420+0000 7f4b4fa00640 1 -- 172.21.15.138:0/2394839599 >> 172.21.15.138:0/2394839599 conn(0x7f4b480cb070 msgr2=0x7f4b4815d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:19.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.421+0000 7f4b4fa00640 1 -- 172.21.15.138:0/2394839599 shutdown_connections 2024-01-21T22:25:19.423 INFO:teuthology.orchestra.run.smithi138.stdout:60129542155 2024-01-21T22:25:19.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.421+0000 7f4b4fa00640 1 -- 172.21.15.138:0/2394839599 wait complete. 2024-01-21T22:25:19.424 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-21T22:25:19.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.427+0000 7f04a2000640 1 -- 172.21.15.138:0/4159512619 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f049402f050 con 0x7f04ac0c9fa0 2024-01-21T22:25:19.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.432+0000 7f3181600640 1 -- 172.21.15.138:0/691470637 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f317c007070 con 0x7f3184153d20 2024-01-21T22:25:19.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.433+0000 7f3181600640 1 --2- 172.21.15.138:0/691470637 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f315c06b380 0x7f315c06d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.429+0000 7f04a2000640 1 --2- 172.21.15.138:0/4159512619 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f048406b380 0x7f048406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.430+0000 7f04a9e00640 1 --2- 172.21.15.138:0/4159512619 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f048406b380 0x7f048406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.430+0000 7f04a2000640 1 -- 172.21.15.138:0/4159512619 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f049407b2e0 con 0x7f04ac0c9fa0 2024-01-21T22:25:19.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.430+0000 7f04a9e00640 1 --2- 172.21.15.138:0/4159512619 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f048406b380 0x7f048406d840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f049c00b810 tx=0x7f049c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.431+0000 7f04a2000640 1 --2- 172.21.15.138:0/4159512619 >> [v2:172.21.15.152:6802/223848127,v1:172.21.15.152:6803/223848127] conn(0x7f0484073520 0x7f0484075940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.431+0000 7f04a2000640 1 -- 172.21.15.138:0/4159512619 --> [v2:172.21.15.152:6802/223848127,v1:172.21.15.152:6803/223848127] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0484075ff0 con 0x7f0484073520 2024-01-21T22:25:19.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.431+0000 7f04aa800640 1 --2- 172.21.15.138:0/4159512619 >> [v2:172.21.15.152:6802/223848127,v1:172.21.15.152:6803/223848127] conn(0x7f0484073520 0x7f0484075940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.432+0000 7f04aa800640 1 --2- 172.21.15.138:0/4159512619 >> [v2:172.21.15.152:6802/223848127,v1:172.21.15.152:6803/223848127] conn(0x7f0484073520 0x7f0484075940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.433+0000 7f04a2000640 1 -- 172.21.15.138:0/4159512619 <== osd.6 v2:172.21.15.152:6802/223848127 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0484075ff0 con 0x7f0484073520 2024-01-21T22:25:19.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.434+0000 7f3181600640 1 -- 172.21.15.138:0/691470637 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f317c058b70 con 0x7f3184153d20 2024-01-21T22:25:19.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.434+0000 7f3189a00640 1 --2- 172.21.15.138:0/691470637 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f315c06b380 0x7f315c06d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.434+0000 7f3181600640 1 --2- 172.21.15.138:0/691470637 >> [v2:172.21.15.138:6800/1365838748,v1:172.21.15.138:6801/1365838748] conn(0x7f315c073520 0x7f315c075940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.434+0000 7f3181600640 1 -- 172.21.15.138:0/691470637 --> [v2:172.21.15.138:6800/1365838748,v1:172.21.15.138:6801/1365838748] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f315c075ff0 con 0x7f315c073520 2024-01-21T22:25:19.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.434+0000 7f3189000640 1 --2- 172.21.15.138:0/691470637 >> [v2:172.21.15.138:6800/1365838748,v1:172.21.15.138:6801/1365838748] conn(0x7f315c073520 0x7f315c075940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.435+0000 7f3189a00640 1 --2- 172.21.15.138:0/691470637 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f315c06b380 0x7f315c06d840 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f316c036000 tx=0x7f316c030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.439 INFO:teuthology.orchestra.run.smithi138.stdout:60129542155 2024-01-21T22:25:19.439 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-01-21T22:25:19.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.436+0000 7f3189000640 1 --2- 172.21.15.138:0/691470637 >> [v2:172.21.15.138:6800/1365838748,v1:172.21.15.138:6801/1365838748] conn(0x7f315c073520 0x7f315c075940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.438+0000 7f3181600640 1 -- 172.21.15.138:0/691470637 <== osd.2 v2:172.21.15.138:6800/1365838748 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f315c075ff0 con 0x7f315c073520 2024-01-21T22:25:19.446 INFO:teuthology.orchestra.run.smithi138.stdout:60129542155 2024-01-21T22:25:19.446 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-21T22:25:19.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.468+0000 7f04ab200640 1 -- 172.21.15.138:0/4159512619 --> [v2:172.21.15.152:6802/223848127,v1:172.21.15.152:6803/223848127] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f04ac159a70 con 0x7f0484073520 2024-01-21T22:25:19.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.469+0000 7f04a2000640 1 -- 172.21.15.138:0/4159512619 <== osd.6 v2:172.21.15.152:6802/223848127 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f04a4009a70 con 0x7f0484073520 2024-01-21T22:25:19.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.474+0000 7f318c400640 1 -- 172.21.15.138:0/691470637 --> [v2:172.21.15.138:6800/1365838748,v1:172.21.15.138:6801/1365838748] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f31840cbb80 con 0x7f315c073520 2024-01-21T22:25:19.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.474+0000 7f3181600640 1 -- 172.21.15.138:0/691470637 <== osd.2 v2:172.21.15.138:6800/1365838748 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3174001500 con 0x7f315c073520 2024-01-21T22:25:19.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.475+0000 7f318c400640 1 -- 172.21.15.138:0/691470637 >> [v2:172.21.15.138:6800/1365838748,v1:172.21.15.138:6801/1365838748] conn(0x7f315c073520 msgr2=0x7f315c075940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.475+0000 7f318c400640 1 --2- 172.21.15.138:0/691470637 >> [v2:172.21.15.138:6800/1365838748,v1:172.21.15.138:6801/1365838748] conn(0x7f315c073520 0x7f315c075940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.475+0000 7f318c400640 1 -- 172.21.15.138:0/691470637 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f315c06b380 msgr2=0x7f315c06d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.475+0000 7f318c400640 1 --2- 172.21.15.138:0/691470637 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f315c06b380 0x7f315c06d840 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f316c036000 tx=0x7f316c030000 comp rx=0 tx=0).stop 2024-01-21T22:25:19.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.475+0000 7f318c400640 1 -- 172.21.15.138:0/691470637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3184153d20 msgr2=0x7f318414a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.475+0000 7f318c400640 1 --2- 172.21.15.138:0/691470637 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3184153d20 0x7f318414a0a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f317c00ca70 tx=0x7f317c00cf30 comp rx=0 tx=0).stop 2024-01-21T22:25:19.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.475+0000 7f318a400640 1 -- 172.21.15.138:0/691470637 reap_dead start 2024-01-21T22:25:19.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.470+0000 7f049b400640 1 -- 172.21.15.138:0/4159512619 >> [v2:172.21.15.152:6802/223848127,v1:172.21.15.152:6803/223848127] conn(0x7f0484073520 msgr2=0x7f0484075940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.470+0000 7f049b400640 1 --2- 172.21.15.138:0/4159512619 >> [v2:172.21.15.152:6802/223848127,v1:172.21.15.152:6803/223848127] conn(0x7f0484073520 0x7f0484075940 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.470+0000 7f049b400640 1 -- 172.21.15.138:0/4159512619 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f048406b380 msgr2=0x7f048406d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.470+0000 7f049b400640 1 --2- 172.21.15.138:0/4159512619 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f048406b380 0x7f048406d840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f049c00b810 tx=0x7f049c008040 comp rx=0 tx=0).stop 2024-01-21T22:25:19.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.470+0000 7f049b400640 1 -- 172.21.15.138:0/4159512619 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04ac0c9fa0 msgr2=0x7f04ac1f0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.470+0000 7f049b400640 1 --2- 172.21.15.138:0/4159512619 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f04ac0c9fa0 0x7f04ac1f0690 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0494033000 tx=0x7f04940046c0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.471+0000 7f04aa800640 1 -- 172.21.15.138:0/4159512619 reap_dead start 2024-01-21T22:25:19.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.472+0000 7f049b400640 1 -- 172.21.15.138:0/4159512619 shutdown_connections 2024-01-21T22:25:19.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.472+0000 7f049b400640 1 -- 172.21.15.138:0/4159512619 >> 172.21.15.138:0/4159512619 conn(0x7f04ac072420 msgr2=0x7f04ac06cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:19.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.472+0000 7f049b400640 1 -- 172.21.15.138:0/4159512619 shutdown_connections 2024-01-21T22:25:19.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.472+0000 7f049b400640 1 -- 172.21.15.138:0/4159512619 wait complete. 2024-01-21T22:25:19.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.469+0000 7fbe36000640 1 -- 172.21.15.138:0/2586780836 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fbe3000a9b0 con 0x7fbe380ca760 2024-01-21T22:25:19.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.470+0000 7fbe36000640 1 --2- 172.21.15.138:0/2586780836 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fbe1006b380 0x7fbe1006d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.472+0000 7fbe3de00640 1 --2- 172.21.15.138:0/2586780836 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fbe1006b380 0x7fbe1006d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.473+0000 7fbe3de00640 1 --2- 172.21.15.138:0/2586780836 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fbe1006b380 0x7fbe1006d840 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009710 tx=0x7fbe28006c90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.478+0000 7f318c400640 1 -- 172.21.15.138:0/691470637 shutdown_connections 2024-01-21T22:25:19.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.478+0000 7f318c400640 1 -- 172.21.15.138:0/691470637 >> 172.21.15.138:0/691470637 conn(0x7f318406d0d0 msgr2=0x7f3184152d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:19.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.478+0000 7f318c400640 1 -- 172.21.15.138:0/691470637 shutdown_connections 2024-01-21T22:25:19.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.479+0000 7f318c400640 1 -- 172.21.15.138:0/691470637 wait complete. 2024-01-21T22:25:19.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.484+0000 7fbe36000640 1 -- 172.21.15.138:0/2586780836 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe300594d0 con 0x7fbe380ca760 2024-01-21T22:25:19.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.485+0000 7fbe36000640 1 --2- 172.21.15.138:0/2586780836 >> [v2:172.21.15.152:6810/52509095,v1:172.21.15.152:6811/52509095] conn(0x7fbe10073520 0x7fbe10075940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.485+0000 7fbe36000640 1 -- 172.21.15.138:0/2586780836 --> [v2:172.21.15.152:6810/52509095,v1:172.21.15.152:6811/52509095] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe10075ff0 con 0x7fbe10073520 2024-01-21T22:25:19.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.485+0000 7fbe36000640 1 -- 172.21.15.138:0/2586780836 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe30055110 con 0x7fbe380ca760 2024-01-21T22:25:19.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.486+0000 7fbe3d400640 1 --2- 172.21.15.138:0/2586780836 >> [v2:172.21.15.152:6810/52509095,v1:172.21.15.152:6811/52509095] conn(0x7fbe10073520 0x7fbe10075940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.488+0000 7fbe3d400640 1 --2- 172.21.15.138:0/2586780836 >> [v2:172.21.15.152:6810/52509095,v1:172.21.15.152:6811/52509095] conn(0x7fbe10073520 0x7fbe10075940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.489+0000 7fbe36000640 1 -- 172.21.15.138:0/2586780836 <== osd.7 v2:172.21.15.152:6810/52509095 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbe10075ff0 con 0x7fbe10073520 2024-01-21T22:25:19.502 INFO:teuthology.orchestra.run.smithi138.stdout:60129542155 2024-01-21T22:25:19.502 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-01-21T22:25:19.507 INFO:teuthology.orchestra.run.smithi138.stdout:60129542154 2024-01-21T22:25:19.507 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-21T22:25:19.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.518+0000 7fbe3f200640 1 -- 172.21.15.138:0/2586780836 --> [v2:172.21.15.152:6810/52509095,v1:172.21.15.152:6811/52509095] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbe380b5390 con 0x7fbe10073520 2024-01-21T22:25:19.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.518+0000 7fbe36000640 1 -- 172.21.15.138:0/2586780836 <== osd.7 v2:172.21.15.152:6810/52509095 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbe380b5390 con 0x7fbe10073520 2024-01-21T22:25:19.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.520+0000 7fbe3f200640 1 -- 172.21.15.138:0/2586780836 >> [v2:172.21.15.152:6810/52509095,v1:172.21.15.152:6811/52509095] conn(0x7fbe10073520 msgr2=0x7fbe10075940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.520+0000 7fbe3f200640 1 --2- 172.21.15.138:0/2586780836 >> [v2:172.21.15.152:6810/52509095,v1:172.21.15.152:6811/52509095] conn(0x7fbe10073520 0x7fbe10075940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.520+0000 7fbe3f200640 1 -- 172.21.15.138:0/2586780836 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fbe1006b380 msgr2=0x7fbe1006d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.520+0000 7fbe3f200640 1 --2- 172.21.15.138:0/2586780836 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fbe1006b380 0x7fbe1006d840 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbe28009710 tx=0x7fbe28006c90 comp rx=0 tx=0).stop 2024-01-21T22:25:19.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.520+0000 7fbe3f200640 1 -- 172.21.15.138:0/2586780836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe380ca760 msgr2=0x7fbe38155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.520+0000 7fbe3f200640 1 --2- 172.21.15.138:0/2586780836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe380ca760 0x7fbe38155930 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000c780 tx=0x7fbe3000cc40 comp rx=0 tx=0).stop 2024-01-21T22:25:19.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.521+0000 7fbe3e800640 1 -- 172.21.15.138:0/2586780836 reap_dead start 2024-01-21T22:25:19.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.522+0000 7fbe3f200640 1 -- 172.21.15.138:0/2586780836 shutdown_connections 2024-01-21T22:25:19.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.522+0000 7fbe3f200640 1 -- 172.21.15.138:0/2586780836 >> 172.21.15.138:0/2586780836 conn(0x7fbe3806ce60 msgr2=0x7fbe3815a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:19.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.522+0000 7fbe3f200640 1 -- 172.21.15.138:0/2586780836 shutdown_connections 2024-01-21T22:25:19.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.523+0000 7fbe3f200640 1 -- 172.21.15.138:0/2586780836 wait complete. 2024-01-21T22:25:19.553 INFO:teuthology.orchestra.run.smithi138.stdout:60129542155 2024-01-21T22:25:19.554 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-01-21T22:25:19.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.672+0000 7f7d1ca00640 1 Processor -- start 2024-01-21T22:25:19.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.672+0000 7f7d1ca00640 1 -- start start 2024-01-21T22:25:19.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.674+0000 7f7d1ca00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7d18152f20 0x7f7d180bfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.674+0000 7f7d1ca00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d180c7540 0x7f7d180c0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.674+0000 7f7d1ca00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d180c7090 con 0x7f7d180c0b20 2024-01-21T22:25:19.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.674+0000 7f7d1ca00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d1806cd40 con 0x7f7d18152f20 2024-01-21T22:25:19.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.674+0000 7f7d1ca00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d18159870 con 0x7f7d180c7540 2024-01-21T22:25:19.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.676+0000 7f7d16a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d180c7540 0x7f7d180c0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.676+0000 7f7d16a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d180c7540 0x7f7d180c0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42368/0 (socket says 172.21.15.138:42368) 2024-01-21T22:25:19.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.676+0000 7f7d16a00640 1 -- 172.21.15.138:0/1718312369 learned_addr learned my addr 172.21.15.138:0/1718312369 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:19.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.678+0000 7f7d16a00640 1 -- 172.21.15.138:0/1718312369 >> v2:172.21.15.138:3300/0 conn(0x7f7d18152f20 msgr2=0x7f7d180bfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:19.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.678+0000 7f7d16a00640 1 --2- 172.21.15.138:0/1718312369 >> v2:172.21.15.138:3300/0 conn(0x7f7d18152f20 0x7f7d180bfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.678+0000 7f7d16a00640 1 -- 172.21.15.138:0/1718312369 >> v1:172.21.15.152:6789/0 conn(0x7f7d180c0b20 legacy=0x7f7d1806bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.678+0000 7f7d16a00640 1 -- 172.21.15.138:0/1718312369 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d18159b10 con 0x7f7d180c7540 2024-01-21T22:25:19.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.682+0000 7f7d16a00640 1 --2- 172.21.15.138:0/1718312369 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d180c7540 0x7f7d180c0ff0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7d08008870 tx=0x7f7d0802e060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f1e87d83e5886a9 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.686+0000 7f7d16000640 1 -- 172.21.15.138:0/1718312369 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0802ee10 con 0x7f7d180c7540 2024-01-21T22:25:19.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.686+0000 7f7d16000640 1 -- 172.21.15.138:0/1718312369 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d08032070 con 0x7f7d180c7540 2024-01-21T22:25:19.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.686+0000 7f7d16000640 1 -- 172.21.15.138:0/1718312369 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d08038620 con 0x7f7d180c7540 2024-01-21T22:25:19.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.687+0000 7f7d1ca00640 1 -- 172.21.15.138:0/1718312369 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d180c7540 msgr2=0x7f7d180c0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.688+0000 7f7d1ca00640 1 --2- 172.21.15.138:0/1718312369 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d180c7540 0x7f7d180c0ff0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7d08008870 tx=0x7f7d0802e060 comp rx=0 tx=0).stop 2024-01-21T22:25:19.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.690+0000 7f7d1ca00640 1 -- 172.21.15.138:0/1718312369 shutdown_connections 2024-01-21T22:25:19.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.690+0000 7f7d1ca00640 1 --2- 172.21.15.138:0/1718312369 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d180c7540 0x7f7d180c0ff0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.690+0000 7f7d1ca00640 1 --2- 172.21.15.138:0/1718312369 >> v2:172.21.15.138:3300/0 conn(0x7f7d18152f20 0x7f7d180bfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.690+0000 7f7d1ca00640 1 -- 172.21.15.138:0/1718312369 >> 172.21.15.138:0/1718312369 conn(0x7f7d1806d930 msgr2=0x7f7d1806dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:19.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.691+0000 7f7d1ca00640 1 -- 172.21.15.138:0/1718312369 shutdown_connections 2024-01-21T22:25:19.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.691+0000 7f7d1ca00640 1 -- 172.21.15.138:0/1718312369 wait complete. 2024-01-21T22:25:19.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.697+0000 7f7d1ca00640 1 Processor -- start 2024-01-21T22:25:19.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.698+0000 7f7d1ca00640 1 -- start start 2024-01-21T22:25:19.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.701+0000 7f0753e00640 1 Processor -- start 2024-01-21T22:25:19.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.702+0000 7f7d1ca00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d180c0b20 0x7f7d18157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.702+0000 7f7d1ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d180c7540 0x7f7d180723b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.702+0000 7f7d1ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d18152f20 0x7f7d181eb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.702+0000 7f7d1ca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d180c4ba0 con 0x7f7d180c7540 2024-01-21T22:25:19.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.702+0000 7f7d1ca00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d18158750 con 0x7f7d18152f20 2024-01-21T22:25:19.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.702+0000 7f7d1ca00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d18158c80 con 0x7f7d180c0b20 2024-01-21T22:25:19.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.703+0000 7f0753e00640 1 -- start start 2024-01-21T22:25:19.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.703+0000 7f7d17400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d180c0b20 0x7f7d18157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.703+0000 7f7d17400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d180c0b20 0x7f7d18157aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42376/0 (socket says 172.21.15.138:42376) 2024-01-21T22:25:19.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.703+0000 7f7d17400640 1 -- 172.21.15.138:0/3599878148 learned_addr learned my addr 172.21.15.138:0/3599878148 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:19.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.704+0000 7f7d17e00640 1 --2- 172.21.15.138:0/3599878148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d18152f20 0x7f7d181eb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.706+0000 7f0753e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f074c04c600 0x7f074c005e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.706+0000 7f0753e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f074c046d70 0x7f074c045330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.706+0000 7f0753e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f074c04b490 con 0x7f074c0d8fb0 2024-01-21T22:25:19.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.706+0000 7f0753e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f074c045fb0 con 0x7f074c04c600 2024-01-21T22:25:19.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.706+0000 7f0753e00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f074c046290 con 0x7f074c046d70 2024-01-21T22:25:19.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.707+0000 7f7d17400640 1 -- 172.21.15.138:0/3599878148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d18152f20 msgr2=0x7f7d181eb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.707+0000 7f7d17400640 1 --2- 172.21.15.138:0/3599878148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d18152f20 0x7f7d181eb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.707+0000 7f7d17400640 1 -- 172.21.15.138:0/3599878148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d180c7540 msgr2=0x7f7d180723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.707+0000 7f7d17400640 1 --2- 172.21.15.138:0/3599878148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d180c7540 0x7f7d180723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.707+0000 7f7d17400640 1 -- 172.21.15.138:0/3599878148 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d08008550 con 0x7f7d180c0b20 2024-01-21T22:25:19.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.710+0000 7f0753400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f074c046d70 0x7f074c045330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.710+0000 7f7d17400640 1 --2- 172.21.15.138:0/3599878148 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d180c0b20 0x7f7d18157aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000ba50 tx=0x7f7d1000bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.710+0000 7f0753400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f074c046d70 0x7f074c045330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42392/0 (socket says 172.21.15.138:42392) 2024-01-21T22:25:19.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.711+0000 7f0753400640 1 -- 172.21.15.138:0/2376586117 learned_addr learned my addr 172.21.15.138:0/2376586117 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:19.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.712+0000 7f0752000640 1 --2- 172.21.15.138:0/2376586117 >> v2:172.21.15.152:3300/0 conn(0x7f074c04c600 0x7f074c005e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.714+0000 7f7d0f400640 1 -- 172.21.15.138:0/3599878148 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1000cbe0 con 0x7f7d180c0b20 2024-01-21T22:25:19.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.714+0000 7f7d0f400640 1 -- 172.21.15.138:0/3599878148 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1000cd40 con 0x7f7d180c0b20 2024-01-21T22:25:19.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.714+0000 7f7d0f400640 1 -- 172.21.15.138:0/3599878148 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1000cbe0 con 0x7f7d180c0b20 2024-01-21T22:25:19.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.715+0000 7f7d1ca00640 1 -- 172.21.15.138:0/3599878148 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d180c22b0 con 0x7f7d180c0b20 2024-01-21T22:25:19.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.715+0000 7f7d1ca00640 1 -- 172.21.15.138:0/3599878148 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d181589a0 con 0x7f7d180c0b20 2024-01-21T22:25:19.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.715+0000 7f0752000640 1 -- 172.21.15.138:0/2376586117 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f074c046d70 msgr2=0x7f074c045330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:19.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.716+0000 7f7d0cc00640 1 -- 172.21.15.138:0/3599878148 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce0005290 con 0x7f7d180c0b20 2024-01-21T22:25:19.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.715+0000 7f0752000640 1 --2- 172.21.15.138:0/2376586117 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f074c046d70 0x7f074c045330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.715+0000 7f0752000640 1 -- 172.21.15.138:0/2376586117 >> v1:172.21.15.138:6789/0 conn(0x7f074c0d8fb0 legacy=0x7f074c0df7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.715+0000 7f0752000640 1 -- 172.21.15.138:0/2376586117 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074c046570 con 0x7f074c04c600 2024-01-21T22:25:19.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.718+0000 7f0752000640 1 --2- 172.21.15.138:0/2376586117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f074c04c600 0x7f074c005e90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f07440097a0 tx=0x7f074402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4eb31c30e734907d server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.721+0000 7f4658a00640 1 Processor -- start 2024-01-21T22:25:19.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.722+0000 7f0751600640 1 -- 172.21.15.138:0/2376586117 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0744031640 con 0x7f074c04c600 2024-01-21T22:25:19.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.722+0000 7f0751600640 1 -- 172.21.15.138:0/2376586117 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07440317a0 con 0x7f074c04c600 2024-01-21T22:25:19.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.722+0000 7f0751600640 1 -- 172.21.15.138:0/2376586117 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0744031b70 con 0x7f074c04c600 2024-01-21T22:25:19.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.721+0000 7f4658a00640 1 -- start start 2024-01-21T22:25:19.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.723+0000 7f4658a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4650153ef0 0x7f46500cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.723+0000 7f4658a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f46500c6620 0x7f46500ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.723+0000 7f4658a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f46500caf80 0x7f465006b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.723+0000 7f4658a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46500ca240 con 0x7f46500caf80 2024-01-21T22:25:19.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.723+0000 7f4658a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f465015aa70 con 0x7f46500c6620 2024-01-21T22:25:19.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.723+0000 7f4658a00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46500caba0 con 0x7f4650153ef0 2024-01-21T22:25:19.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.724+0000 7f4655600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f46500c6620 0x7f46500ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.724+0000 7f0753e00640 1 -- 172.21.15.138:0/2376586117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f074c04c600 msgr2=0x7f074c005e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.724+0000 7f0753e00640 1 --2- 172.21.15.138:0/2376586117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f074c04c600 0x7f074c005e90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f07440097a0 tx=0x7f074402b8f0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.724+0000 7f4655600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f46500c6620 0x7f46500ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:36200/0 (socket says 172.21.15.138:36200) 2024-01-21T22:25:19.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.724+0000 7f4655600640 1 -- 172.21.15.138:0/868856378 learned_addr learned my addr 172.21.15.138:0/868856378 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:19.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.724+0000 7f4656a00640 1 --2- 172.21.15.138:0/868856378 >> v2:172.21.15.138:3300/0 conn(0x7f46500caf80 0x7f465006b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.724+0000 7f0753e00640 1 -- 172.21.15.138:0/2376586117 shutdown_connections 2024-01-21T22:25:19.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.724+0000 7f0753e00640 1 --2- 172.21.15.138:0/2376586117 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f074c046d70 0x7f074c045330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.724+0000 7f0753e00640 1 --2- 172.21.15.138:0/2376586117 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f074c04c600 0x7f074c005e90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.724+0000 7f0753e00640 1 -- 172.21.15.138:0/2376586117 >> 172.21.15.138:0/2376586117 conn(0x7f074c050180 msgr2=0x7f074c050590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:19.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.725+0000 7f0753e00640 1 -- 172.21.15.138:0/2376586117 shutdown_connections 2024-01-21T22:25:19.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.725+0000 7f0753e00640 1 -- 172.21.15.138:0/2376586117 wait complete. 2024-01-21T22:25:19.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.727+0000 7f4655600640 1 -- 172.21.15.138:0/868856378 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4650153ef0 msgr2=0x7f46500cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:19.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.727+0000 7f4655600640 1 --2- 172.21.15.138:0/868856378 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4650153ef0 0x7f46500cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.728+0000 7f0753e00640 1 Processor -- start 2024-01-21T22:25:19.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.728+0000 7f0753e00640 1 -- start start 2024-01-21T22:25:19.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.727+0000 7f4655600640 1 -- 172.21.15.138:0/868856378 >> v2:172.21.15.138:3300/0 conn(0x7f46500caf80 msgr2=0x7f465006b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:19.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.727+0000 7f4655600640 1 --2- 172.21.15.138:0/868856378 >> v2:172.21.15.138:3300/0 conn(0x7f46500caf80 0x7f465006b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.727+0000 7f4655600640 1 -- 172.21.15.138:0/868856378 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46500cb700 con 0x7f46500c6620 2024-01-21T22:25:19.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.729+0000 7f0753e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f074c046d70 0x7f074c0d3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.730+0000 7f4655600640 1 --2- 172.21.15.138:0/868856378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46500c6620 0x7f46500ca660 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4650069bb0 tx=0x7f463802b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=492e0b7fb736388 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.730+0000 7f0753e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f074c04c600 0x7f074c16e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.730+0000 7f0753e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f074c0d8fb0 0x7f074c0d17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.730+0000 7f0753e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c0d08e0 con 0x7f074c046d70 2024-01-21T22:25:19.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.730+0000 7f0753e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c0d58d0 con 0x7f074c0d8fb0 2024-01-21T22:25:19.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.730+0000 7f0753e00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f074c0d5310 con 0x7f074c04c600 2024-01-21T22:25:19.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.731+0000 7f0753400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f074c0d8fb0 0x7f074c0d17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.732+0000 7f0753400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f074c0d8fb0 0x7f074c0d17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:36214/0 (socket says 172.21.15.138:36214) 2024-01-21T22:25:19.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.732+0000 7f0753400640 1 -- 172.21.15.138:0/2370767655 learned_addr learned my addr 172.21.15.138:0/2370767655 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:19.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.733+0000 7f4654c00640 1 -- 172.21.15.138:0/868856378 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4638030890 con 0x7f46500c6620 2024-01-21T22:25:19.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.733+0000 7f0752a00640 1 --2- 172.21.15.138:0/2370767655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f074c046d70 0x7f074c0d3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.735+0000 7f4654c00640 1 -- 172.21.15.138:0/868856378 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46380309f0 con 0x7f46500c6620 2024-01-21T22:25:19.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.735+0000 7f4654c00640 1 -- 172.21.15.138:0/868856378 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4638030d80 con 0x7f46500c6620 2024-01-21T22:25:19.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.736+0000 7f4658a00640 1 -- 172.21.15.138:0/868856378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46500c6620 msgr2=0x7f46500ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.736+0000 7f4658a00640 1 --2- 172.21.15.138:0/868856378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46500c6620 0x7f46500ca660 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4650069bb0 tx=0x7f463802b9d0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.736+0000 7f4658a00640 1 -- 172.21.15.138:0/868856378 shutdown_connections 2024-01-21T22:25:19.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.736+0000 7f4658a00640 1 --2- 172.21.15.138:0/868856378 >> v2:172.21.15.138:3300/0 conn(0x7f46500caf80 0x7f465006b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.736+0000 7f4658a00640 1 --2- 172.21.15.138:0/868856378 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46500c6620 0x7f46500ca660 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.736+0000 7f4658a00640 1 --2- 172.21.15.138:0/868856378 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4650153ef0 0x7f46500cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.736+0000 7f4658a00640 1 -- 172.21.15.138:0/868856378 >> 172.21.15.138:0/868856378 conn(0x7f46500cbca0 msgr2=0x7f46500cc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:19.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.737+0000 7f4658a00640 1 -- 172.21.15.138:0/868856378 shutdown_connections 2024-01-21T22:25:19.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.737+0000 7f4658a00640 1 -- 172.21.15.138:0/868856378 wait complete. 2024-01-21T22:25:19.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.738+0000 7fcdf4400640 1 Processor -- start 2024-01-21T22:25:19.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.738+0000 7fcdf4400640 1 -- start start 2024-01-21T22:25:19.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.739+0000 7f4658a00640 1 Processor -- start 2024-01-21T22:25:19.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.740+0000 7f4658a00640 1 -- start start 2024-01-21T22:25:19.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.740+0000 7fcdf4400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcdec154e00 0x7fcdec161e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.740+0000 7fcdf4400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcdec0cc7c0 0x7fcdec06b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.740+0000 7fcdf4400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcdec06bd40 0x7fcdec0c7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.740+0000 7fcdf4400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdec0c69d0 con 0x7fcdec06bd40 2024-01-21T22:25:19.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.740+0000 7fcdf4400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdec1551e0 con 0x7fcdec154e00 2024-01-21T22:25:19.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.740+0000 7fcdf4400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec0cac30 con 0x7fcdec0cc7c0 2024-01-21T22:25:19.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.740+0000 7f0753400640 1 -- 172.21.15.138:0/2370767655 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f074c04c600 msgr2=0x7f074c16e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.740+0000 7f0753400640 1 --2- 172.21.15.138:0/2370767655 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f074c04c600 0x7f074c16e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.740+0000 7f0753400640 1 -- 172.21.15.138:0/2370767655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f074c046d70 msgr2=0x7f074c0d3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.740+0000 7f0753400640 1 --2- 172.21.15.138:0/2370767655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f074c046d70 0x7f074c0d3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.740+0000 7f0753400640 1 -- 172.21.15.138:0/2370767655 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0744009400 con 0x7f074c0d8fb0 2024-01-21T22:25:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.741+0000 7f4658a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46500c6620 0x7f46500c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.741+0000 7f4658a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f46500caf80 0x7f46500c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.741+0000 7f4658a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4650153ef0 0x7f46500c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.742+0000 7f4658a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46501574e0 con 0x7f4650153ef0 2024-01-21T22:25:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.742+0000 7f4658a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465006b9e0 con 0x7f46500c6620 2024-01-21T22:25:19.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.742+0000 7f4658a00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46500c4400 con 0x7f46500caf80 2024-01-21T22:25:19.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.744+0000 7f4656000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46500c6620 0x7f46500c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.744+0000 7fcdf1a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcdec154e00 0x7fcdec161e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.745+0000 7f0753400640 1 --2- 172.21.15.138:0/2370767655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f074c0d8fb0 0x7f074c0d17b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f073c00e870 tx=0x7f073c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.745+0000 7f4656000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46500c6620 0x7f46500c3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:36222/0 (socket says 172.21.15.138:36222) 2024-01-21T22:25:19.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.745+0000 7f4656000640 1 -- 172.21.15.138:0/4045529249 learned_addr learned my addr 172.21.15.138:0/4045529249 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:19.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.745+0000 7fcdf2400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcdec06bd40 0x7fcdec0c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.745+0000 7fcdf1a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcdec154e00 0x7fcdec161e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:36220/0 (socket says 172.21.15.138:36220) 2024-01-21T22:25:19.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.745+0000 7fcdf1a00640 1 -- 172.21.15.138:0/2939481060 learned_addr learned my addr 172.21.15.138:0/2939481060 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:19.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.746+0000 7f4656a00640 1 --2- 172.21.15.138:0/4045529249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4650153ef0 0x7f46500c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.746+0000 7f7d0f400640 1 -- 172.21.15.138:0/3599878148 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f7d10007370 con 0x7f7d180c0b20 2024-01-21T22:25:19.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.747+0000 7f7d0f400640 1 --2- 172.21.15.138:0/3599878148 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7cf006b3c0 0x7f7cf006d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.748+0000 7f7d16a00640 1 --2- 172.21.15.138:0/3599878148 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7cf006b3c0 0x7f7cf006d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.748+0000 7fcdf2400640 1 -- 172.21.15.138:0/2939481060 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcdec0cc7c0 msgr2=0x7fcdec06b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:19.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.748+0000 7fcdf2400640 1 --2- 172.21.15.138:0/2939481060 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcdec0cc7c0 0x7fcdec06b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.748+0000 7fcdf2400640 1 -- 172.21.15.138:0/2939481060 >> v2:172.21.15.152:3300/0 conn(0x7fcdec154e00 msgr2=0x7fcdec161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:19.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.748+0000 7fcdf2400640 1 --2- 172.21.15.138:0/2939481060 >> v2:172.21.15.152:3300/0 conn(0x7fcdec154e00 0x7fcdec161e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.748+0000 7fcdf2400640 1 -- 172.21.15.138:0/2939481060 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec1623c0 con 0x7fcdec06bd40 2024-01-21T22:25:19.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.749+0000 7f7d16a00640 1 --2- 172.21.15.138:0/3599878148 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7cf006b3c0 0x7f7cf006d880 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7d08039000 tx=0x7f7d08008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.754+0000 7f4656000640 1 -- 172.21.15.138:0/4045529249 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f46500caf80 msgr2=0x7f46500c54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.754+0000 7f4656000640 1 --2- 172.21.15.138:0/4045529249 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f46500caf80 0x7f46500c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.754+0000 7f4656000640 1 -- 172.21.15.138:0/4045529249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4650153ef0 msgr2=0x7f46500c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.754+0000 7f4656000640 1 --2- 172.21.15.138:0/4045529249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4650153ef0 0x7f46500c59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.754+0000 7f4656000640 1 -- 172.21.15.138:0/4045529249 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46380094d0 con 0x7f46500c6620 2024-01-21T22:25:19.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.754+0000 7f074aa00640 1 -- 172.21.15.138:0/2370767655 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c00ee60 con 0x7f074c0d8fb0 2024-01-21T22:25:19.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.755+0000 7f074aa00640 1 -- 172.21.15.138:0/2370767655 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f073c00c940 con 0x7f074c0d8fb0 2024-01-21T22:25:19.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.755+0000 7f074aa00640 1 -- 172.21.15.138:0/2370767655 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c00ccf0 con 0x7f074c0d8fb0 2024-01-21T22:25:19.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.755+0000 7f0753e00640 1 -- 172.21.15.138:0/2370767655 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f074c0d55f0 con 0x7f074c0d8fb0 2024-01-21T22:25:19.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.755+0000 7f0753e00640 1 -- 172.21.15.138:0/2370767655 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f074c0d0fb0 con 0x7f074c0d8fb0 2024-01-21T22:25:19.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.757+0000 7f0753e00640 1 -- 172.21.15.138:0/2370767655 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f074c005390 con 0x7f074c0d8fb0 2024-01-21T22:25:19.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.757+0000 7fcdf2400640 1 --2- 172.21.15.138:0/2939481060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdec06bd40 0x7fcdec0c7840 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcdec06c860 tx=0x7fcddc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100e864d92eccd06 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.762+0000 7fcdebe00640 1 -- 172.21.15.138:0/2939481060 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc02e070 con 0x7fcdec06bd40 2024-01-21T22:25:19.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.765+0000 7fcdebe00640 1 -- 172.21.15.138:0/2939481060 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcddc0315a0 con 0x7fcdec06bd40 2024-01-21T22:25:19.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.765+0000 7fcdebe00640 1 -- 172.21.15.138:0/2939481060 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc031930 con 0x7fcdec06bd40 2024-01-21T22:25:19.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.766+0000 7fcdf4400640 1 -- 172.21.15.138:0/2939481060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdec06bd40 msgr2=0x7fcdec0c7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.766+0000 7fcdf4400640 1 --2- 172.21.15.138:0/2939481060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdec06bd40 0x7fcdec0c7840 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcdec06c860 tx=0x7fcddc02b840 comp rx=0 tx=0).stop 2024-01-21T22:25:19.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.766+0000 7fcdf4400640 1 -- 172.21.15.138:0/2939481060 shutdown_connections 2024-01-21T22:25:19.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.766+0000 7fcdf4400640 1 --2- 172.21.15.138:0/2939481060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdec06bd40 0x7fcdec0c7840 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.766+0000 7fcdf4400640 1 --2- 172.21.15.138:0/2939481060 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcdec0cc7c0 0x7fcdec06b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.766+0000 7fcdf4400640 1 --2- 172.21.15.138:0/2939481060 >> v2:172.21.15.152:3300/0 conn(0x7fcdec154e00 0x7fcdec161e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.766+0000 7fcdf4400640 1 -- 172.21.15.138:0/2939481060 >> 172.21.15.138:0/2939481060 conn(0x7fcdec0cbca0 msgr2=0x7fcdec0cc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:19.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.767+0000 7fcdf4400640 1 -- 172.21.15.138:0/2939481060 shutdown_connections 2024-01-21T22:25:19.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.767+0000 7fcdf4400640 1 -- 172.21.15.138:0/2939481060 wait complete. 2024-01-21T22:25:19.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.769+0000 7fcdf4400640 1 Processor -- start 2024-01-21T22:25:19.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.769+0000 7fcdf4400640 1 -- start start 2024-01-21T22:25:19.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.770+0000 7fcdf4400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcdec06bd40 0x7fcdec0c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.770+0000 7fcdf4400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcdec0cc7c0 0x7fcdec0c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.770+0000 7fcdf4400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdec154e00 0x7fcdec1f3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.770+0000 7fcdf4400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec1f7d90 con 0x7fcdec154e00 2024-01-21T22:25:19.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.770+0000 7fcdf4400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec1f6f30 con 0x7fcdec06bd40 2024-01-21T22:25:19.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.770+0000 7fcdf4400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec1f7210 con 0x7fcdec0cc7c0 2024-01-21T22:25:19.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.772+0000 7fcdf2400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdec154e00 0x7fcdec1f3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.772+0000 7fcdf2400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdec154e00 0x7fcdec1f3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52988/0 (socket says 172.21.15.138:52988) 2024-01-21T22:25:19.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.772+0000 7fcdf2400640 1 -- 172.21.15.138:0/2872930786 learned_addr learned my addr 172.21.15.138:0/2872930786 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:19.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.776+0000 7fcdf2400640 1 -- 172.21.15.138:0/2872930786 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcdec0cc7c0 msgr2=0x7fcdec0c5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.776+0000 7fcdf2400640 1 --2- 172.21.15.138:0/2872930786 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcdec0cc7c0 0x7fcdec0c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.776+0000 7fcdf2400640 1 -- 172.21.15.138:0/2872930786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcdec06bd40 msgr2=0x7fcdec0c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:19.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.776+0000 7fcdf2400640 1 --2- 172.21.15.138:0/2872930786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcdec06bd40 0x7fcdec0c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:19.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.776+0000 7fcdf2400640 1 -- 172.21.15.138:0/2872930786 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc009400 con 0x7fcdec154e00 2024-01-21T22:25:19.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.779+0000 7fcdf2400640 1 --2- 172.21.15.138:0/2872930786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdec154e00 0x7fcdec1f3cc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcddc02b880 tx=0x7fcddc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:19.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.780+0000 7f4af5a00640 1 Processor -- start 2024-01-21T22:25:19.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.780+0000 7f4af5a00640 1 -- start start 2024-01-21T22:25:19.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.782+0000 7f4af5a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4af00c6620 0x7f4af00cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.782+0000 7f4af5a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4af00caf80 0x7f4af00ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.782+0000 7f4af5a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4af00caba0 con 0x7f4af0153ef0 2024-01-21T22:25:19.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.782+0000 7f4af5a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4af00ca240 con 0x7f4af00c6620 2024-01-21T22:25:19.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.782+0000 7f4af5a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4af015aa70 con 0x7f4af00caf80 2024-01-21T22:25:19.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.784+0000 7fcde9600640 1 -- 172.21.15.138:0/2872930786 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc032840 con 0x7fcdec154e00 2024-01-21T22:25:19.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.784+0000 7fcde9600640 1 -- 172.21.15.138:0/2872930786 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcddc0329c0 con 0x7fcdec154e00 2024-01-21T22:25:19.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.784+0000 7fcde9600640 1 -- 172.21.15.138:0/2872930786 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc032d70 con 0x7fcdec154e00 2024-01-21T22:25:19.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.784+0000 7fcdf4400640 1 -- 172.21.15.138:0/2872930786 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdec1f74f0 con 0x7fcdec154e00 2024-01-21T22:25:19.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.784+0000 7fcdf4400640 1 -- 172.21.15.138:0/2872930786 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdec1f7ab0 con 0x7fcdec154e00 2024-01-21T22:25:19.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.785+0000 7f4aee000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4af00c6620 0x7f4af00cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:19.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.785+0000 7fcdf4400640 1 -- 172.21.15.138:0/2872930786 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdec04ea30 con 0x7fcdec154e00 2024-01-21T22:25:19.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.786+0000 7f4aee000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f4af00c6620 0x7f4af00cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53004/0 (socket says 172.21.15.138:53004) 2024-01-21T22:25:19.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.786+0000 7f4aee000640 1 -- 172.21.15.138:0/24667861 learned_addr learned my addr 172.21.15.138:0/24667861 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:19.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.798+0000 7f38e8e00640 1 Processor -- start 2024-01-21T22:25:19.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.798+0000 7f38e8e00640 1 -- start start 2024-01-21T22:25:19.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.800+0000 7f38e8e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38e4153ef0 0x7f38e40cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.800+0000 7f38e8e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f38e40c6620 0x7f38e40ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.800+0000 7f38e8e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e40caba0 con 0x7f38e40caf80 2024-01-21T22:25:19.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.800+0000 7f38e8e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e40ca240 con 0x7f38e40c6620 2024-01-21T22:25:19.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.800+0000 7f38e8e00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e415aa70 con 0x7f38e4153ef0 2024-01-21T22:25:19.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.813+0000 7f8401a00640 1 Processor -- start 2024-01-21T22:25:19.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.813+0000 7f8401a00640 1 -- start start 2024-01-21T22:25:19.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.814+0000 7f8401a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f83fc153c60 0x7f83fc0cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.814+0000 7f8401a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83fc154040 0x7f83fc0cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.814+0000 7f8401a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83fc0cbb30 con 0x7f83fc0c4500 2024-01-21T22:25:19.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.814+0000 7f8401a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83fc15a970 con 0x7f83fc153c60 2024-01-21T22:25:19.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.814+0000 7f8401a00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc15dfe0 con 0x7f83fc154040 2024-01-21T22:25:19.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.850+0000 7f3233200640 1 Processor -- start 2024-01-21T22:25:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.850+0000 7f3233200640 1 -- start start 2024-01-21T22:25:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.851+0000 7f3233200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f322c06b9b0 0x7f322c158280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.851+0000 7f3233200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f322c0c5970 0x7f322c161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:19.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.852+0000 7f3233200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322c06be80 con 0x7f322c157ea0 2024-01-21T22:25:19.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.852+0000 7f3233200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f322c15dfe0 con 0x7f322c06b9b0 2024-01-21T22:25:19.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:19.852+0000 7f3233200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f322c15e2c0 con 0x7f322c0c5970 2024-01-21T22:25:20.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.002+0000 7f7d0f400640 1 -- 172.21.15.138:0/3599878148 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d10058570 con 0x7f7d180c0b20 2024-01-21T22:25:20.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.020+0000 7f7d0f400640 1 -- 172.21.15.138:0/3599878148 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f7d100548b0 con 0x7f7d180c0b20 2024-01-21T22:25:20.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.022+0000 7f38e1e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38e4153ef0 0x7f38e40cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:20.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.024+0000 7f38e1e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38e4153ef0 0x7f38e40cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42444/0 (socket says 172.21.15.138:42444) 2024-01-21T22:25:20.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.024+0000 7f38e1e00640 1 -- 172.21.15.138:0/3522995849 learned_addr learned my addr 172.21.15.138:0/3522995849 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:20.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.025+0000 7f83fa000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83fc154040 0x7f83fc0cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:20.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.026+0000 7f4aee000640 1 -- 172.21.15.138:0/24667861 >> v2:172.21.15.152:3300/0 conn(0x7f4af00caf80 msgr2=0x7f4af00ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:20.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.026+0000 7f4aee000640 1 --2- 172.21.15.138:0/24667861 >> v2:172.21.15.152:3300/0 conn(0x7f4af00caf80 0x7f4af00ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.026+0000 7f4aee000640 1 -- 172.21.15.138:0/24667861 >> v1:172.21.15.138:6789/0 conn(0x7f4af0153ef0 legacy=0x7f4af006b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:20.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.026+0000 7f4aee000640 1 -- 172.21.15.138:0/24667861 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af00cb700 con 0x7f4af00c6620 2024-01-21T22:25:20.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.027+0000 7f83fa000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83fc154040 0x7f83fc0cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42446/0 (socket says 172.21.15.138:42446) 2024-01-21T22:25:20.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.027+0000 7f83fa000640 1 -- 172.21.15.138:0/744433484 learned_addr learned my addr 172.21.15.138:0/744433484 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:20.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.031+0000 7f38e1e00640 1 -- 172.21.15.138:0/3522995849 >> v2:172.21.15.152:3300/0 conn(0x7f38e40c6620 msgr2=0x7f38e40ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:20.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.031+0000 7f38e1e00640 1 --2- 172.21.15.138:0/3522995849 >> v2:172.21.15.152:3300/0 conn(0x7f38e40c6620 0x7f38e40ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.031+0000 7f38e1e00640 1 -- 172.21.15.138:0/3522995849 >> v1:172.21.15.138:6789/0 conn(0x7f38e40caf80 legacy=0x7f38e406b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:20.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.031+0000 7f38e1e00640 1 -- 172.21.15.138:0/3522995849 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e40cb700 con 0x7f38e4153ef0 2024-01-21T22:25:20.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.035+0000 7f83fa000640 1 -- 172.21.15.138:0/744433484 >> v2:172.21.15.152:3300/0 conn(0x7f83fc153c60 msgr2=0x7f83fc0cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:20.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.036+0000 7f83fa000640 1 --2- 172.21.15.138:0/744433484 >> v2:172.21.15.152:3300/0 conn(0x7f83fc153c60 0x7f83fc0cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.036+0000 7f83fa000640 1 -- 172.21.15.138:0/744433484 >> v1:172.21.15.152:6789/0 conn(0x7f83fc0c4500 legacy=0x7f83fc0c4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:20.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.036+0000 7f83fa000640 1 -- 172.21.15.138:0/744433484 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc159cf0 con 0x7f83fc154040 2024-01-21T22:25:20.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.039+0000 7f38e1e00640 1 --2- 172.21.15.138:0/3522995849 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38e4153ef0 0x7f38e40cc7c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009870 tx=0x7f38d402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca27e45d59673b61 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:20.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.041+0000 7f4656000640 1 --2- 172.21.15.138:0/4045529249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46500c6620 0x7f46500c3ec0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f464000b410 tx=0x7f464000b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:20.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.042+0000 7f074aa00640 1 -- 172.21.15.138:0/2370767655 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f073c013470 con 0x7f074c0d8fb0 2024-01-21T22:25:20.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.043+0000 7f38e0a00640 1 -- 172.21.15.138:0/3522995849 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4030840 con 0x7f38e4153ef0 2024-01-21T22:25:20.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.043+0000 7f38e0a00640 1 -- 172.21.15.138:0/3522995849 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d40309a0 con 0x7f38e4153ef0 2024-01-21T22:25:20.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.043+0000 7f38e0a00640 1 -- 172.21.15.138:0/3522995849 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4030d30 con 0x7f38e4153ef0 2024-01-21T22:25:20.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.044+0000 7f83fa000640 1 --2- 172.21.15.138:0/744433484 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83fc154040 0x7f83fc0cc270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009870 tx=0x7f83e402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=597999c096d55585 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:20.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.044+0000 7f38e8e00640 1 -- 172.21.15.138:0/3522995849 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38e4153ef0 msgr2=0x7f38e40cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.044+0000 7f38e8e00640 1 --2- 172.21.15.138:0/3522995849 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38e4153ef0 0x7f38e40cc7c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009870 tx=0x7f38d402b980 comp rx=0 tx=0).stop 2024-01-21T22:25:20.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.044+0000 7f38e8e00640 1 -- 172.21.15.138:0/3522995849 shutdown_connections 2024-01-21T22:25:20.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.044+0000 7f38e8e00640 1 --2- 172.21.15.138:0/3522995849 >> v2:172.21.15.152:3300/0 conn(0x7f38e40c6620 0x7f38e40ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.044+0000 7f38e8e00640 1 --2- 172.21.15.138:0/3522995849 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38e4153ef0 0x7f38e40cc7c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.044+0000 7f38e8e00640 1 -- 172.21.15.138:0/3522995849 >> 172.21.15.138:0/3522995849 conn(0x7f38e40cbca0 msgr2=0x7f38e40cc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:20.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.044+0000 7f074aa00640 1 --2- 172.21.15.138:0/2370767655 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f072c06b380 0x7f072c06d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:20.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.044+0000 7f38e8e00640 1 -- 172.21.15.138:0/3522995849 shutdown_connections 2024-01-21T22:25:20.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.045+0000 7f38e8e00640 1 -- 172.21.15.138:0/3522995849 wait complete. 2024-01-21T22:25:20.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.045+0000 7f0752a00640 1 --2- 172.21.15.138:0/2370767655 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f072c06b380 0x7f072c06d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:20.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.045+0000 7f0752a00640 1 --2- 172.21.15.138:0/2370767655 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f072c06b380 0x7f072c06d840 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0754050330 tx=0x7f07540690f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:20.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.047+0000 7f38e8e00640 1 Processor -- start 2024-01-21T22:25:20.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.048+0000 7f38e8e00640 1 -- start start 2024-01-21T22:25:20.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.048+0000 7f83f9600640 1 -- 172.21.15.138:0/744433484 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e4030840 con 0x7f83fc154040 2024-01-21T22:25:20.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.048+0000 7f074aa00640 1 -- 172.21.15.138:0/2370767655 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f073c058df0 con 0x7f074c0d8fb0 2024-01-21T22:25:20.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.048+0000 7f83f9600640 1 -- 172.21.15.138:0/744433484 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e40309a0 con 0x7f83fc154040 2024-01-21T22:25:20.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.048+0000 7f83f9600640 1 -- 172.21.15.138:0/744433484 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e4030d30 con 0x7f83fc154040 2024-01-21T22:25:20.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.049+0000 7fcde9600640 1 -- 172.21.15.138:0/2872930786 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fcddc0453f0 con 0x7fcdec154e00 2024-01-21T22:25:20.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.049+0000 7f8401a00640 1 -- 172.21.15.138:0/744433484 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83fc154040 msgr2=0x7f83fc0cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.049+0000 7f8401a00640 1 --2- 172.21.15.138:0/744433484 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83fc154040 0x7f83fc0cc270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009870 tx=0x7f83e402b980 comp rx=0 tx=0).stop 2024-01-21T22:25:20.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.049+0000 7f38e8e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38e40c6620 0x7f38e40c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:20.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.049+0000 7f8401a00640 1 -- 172.21.15.138:0/744433484 shutdown_connections 2024-01-21T22:25:20.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.049+0000 7f8401a00640 1 --2- 172.21.15.138:0/744433484 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83fc154040 0x7f83fc0cc270 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.049+0000 7f8401a00640 1 --2- 172.21.15.138:0/744433484 >> v2:172.21.15.152:3300/0 conn(0x7f83fc153c60 0x7f83fc0cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.049+0000 7f8401a00640 1 -- 172.21.15.138:0/744433484 >> 172.21.15.138:0/744433484 conn(0x7f83fc06b630 msgr2=0x7f83fc06cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:20.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.049+0000 7f38e8e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38e40caf80 0x7f38e40c3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:20.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.050+0000 7f8401a00640 1 -- 172.21.15.138:0/744433484 shutdown_connections 2024-01-21T22:25:20.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.049+0000 7f38e8e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38e4153ef0 0x7f38e4146de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:20.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.049+0000 7f38e8e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e4145a90 con 0x7f38e40c6620 2024-01-21T22:25:20.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.049+0000 7f38e8e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e40c5d00 con 0x7f38e4153ef0 2024-01-21T22:25:20.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.049+0000 7f38e8e00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e41537c0 con 0x7f38e40caf80 2024-01-21T22:25:20.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.050+0000 7fcde9600640 1 --2- 172.21.15.138:0/2872930786 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcdc406b380 0x7fcdc406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:20.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.050+0000 7fcdf1a00640 1 --2- 172.21.15.138:0/2872930786 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcdc406b380 0x7fcdc406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:20.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.050+0000 7f8401a00640 1 -- 172.21.15.138:0/744433484 wait complete. 2024-01-21T22:25:20.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.051+0000 7fcdf1a00640 1 --2- 172.21.15.138:0/2872930786 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcdc406b380 0x7fcdc406d840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009fa0 tx=0x7fcde400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:20.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.051+0000 7f38e1400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38e40caf80 0x7f38e40c3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:20.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.051+0000 7f8401a00640 1 Processor -- start 2024-01-21T22:25:20.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.051+0000 7f8401a00640 1 -- start start 2024-01-21T22:25:20.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.052+0000 7f38e1400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38e40caf80 0x7f38e40c3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42450/0 (socket says 172.21.15.138:42450) 2024-01-21T22:25:20.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.052+0000 7f38e1400640 1 -- 172.21.15.138:0/1139384074 learned_addr learned my addr 172.21.15.138:0/1139384074 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:20.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.052+0000 7f8401a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83fc0c4500 0x7f83fc1f7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:20.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.052+0000 7f8401a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83fc153c60 0x7f83fc0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:20.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.052+0000 7f8401a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83fc154040 0x7f83fc0c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:20.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.052+0000 7f8401a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc0ce030 con 0x7f83fc0c4500 2024-01-21T22:25:20.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.052+0000 7f8401a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc0ce310 con 0x7f83fc153c60 2024-01-21T22:25:20.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.052+0000 7f8401a00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc0c6d50 con 0x7f83fc154040 2024-01-21T22:25:20.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.055+0000 7f38e1400640 1 -- 172.21.15.138:0/1139384074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38e4153ef0 msgr2=0x7f38e4146de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.055+0000 7f38e1400640 1 --2- 172.21.15.138:0/1139384074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38e4153ef0 0x7f38e4146de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.055+0000 7f38e1400640 1 -- 172.21.15.138:0/1139384074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38e40c6620 msgr2=0x7f38e40c2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.055+0000 7f38e1400640 1 --2- 172.21.15.138:0/1139384074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38e40c6620 0x7f38e40c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.055+0000 7f38e1400640 1 -- 172.21.15.138:0/1139384074 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d40094d0 con 0x7f38e40caf80 2024-01-21T22:25:20.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.057+0000 7f83fb400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83fc154040 0x7f83fc0c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:20.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.058+0000 7f38e1400640 1 --2- 172.21.15.138:0/1139384074 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38e40caf80 0x7f38e40c3460 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00b4d0 tx=0x7f38cc00b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:20.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.059+0000 7f83fb400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83fc154040 0x7f83fc0c66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42454/0 (socket says 172.21.15.138:42454) 2024-01-21T22:25:20.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.059+0000 7f83fb400640 1 -- 172.21.15.138:0/2092804195 learned_addr learned my addr 172.21.15.138:0/2092804195 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:20.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.060+0000 7f074aa00640 1 -- 172.21.15.138:0/2370767655 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f073c055130 con 0x7f074c0d8fb0 2024-01-21T22:25:20.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.063+0000 7f83fb400640 1 -- 172.21.15.138:0/2092804195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83fc153c60 msgr2=0x7f83fc0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.063+0000 7f83fb400640 1 --2- 172.21.15.138:0/2092804195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83fc153c60 0x7f83fc0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.063+0000 7f83fb400640 1 -- 172.21.15.138:0/2092804195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83fc0c4500 msgr2=0x7f83fc1f7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.063+0000 7f83fb400640 1 --2- 172.21.15.138:0/2092804195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83fc0c4500 0x7f83fc1f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.063+0000 7f83fb400640 1 -- 172.21.15.138:0/2092804195 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e40094d0 con 0x7f83fc154040 2024-01-21T22:25:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.065+0000 7f38da000640 1 -- 172.21.15.138:0/1139384074 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc010870 con 0x7f38e40caf80 2024-01-21T22:25:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.065+0000 7f38e8e00640 1 -- 172.21.15.138:0/1139384074 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e4153520 con 0x7f38e40caf80 2024-01-21T22:25:20.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.065+0000 7f38e8e00640 1 -- 172.21.15.138:0/1139384074 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e40c5640 con 0x7f38e40caf80 2024-01-21T22:25:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.066+0000 7f464e000640 1 -- 172.21.15.138:0/4045529249 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4640011830 con 0x7f46500c6620 2024-01-21T22:25:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.066+0000 7f464e000640 1 -- 172.21.15.138:0/4045529249 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46400119b0 con 0x7f46500c6620 2024-01-21T22:25:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.066+0000 7f38e8e00640 1 -- 172.21.15.138:0/1139384074 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e404ea30 con 0x7f38e40caf80 2024-01-21T22:25:20.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.067+0000 7f464e000640 1 -- 172.21.15.138:0/4045529249 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4640011d60 con 0x7f46500c6620 2024-01-21T22:25:20.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.067+0000 7f4658a00640 1 -- 172.21.15.138:0/4045529249 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46500c5f30 con 0x7f46500c6620 2024-01-21T22:25:20.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.067+0000 7f4658a00640 1 -- 172.21.15.138:0/4045529249 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46500c2760 con 0x7f46500c6620 2024-01-21T22:25:20.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.067+0000 7f38da000640 1 -- 172.21.15.138:0/1139384074 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38cc0092b0 con 0x7f38e40caf80 2024-01-21T22:25:20.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.067+0000 7f38da000640 1 -- 172.21.15.138:0/1139384074 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc010bd0 con 0x7f38e40caf80 2024-01-21T22:25:20.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.068+0000 7f83fb400640 1 --2- 172.21.15.138:0/2092804195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83fc154040 0x7f83fc0c66a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f83f400c780 tx=0x7f83f400cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:20.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.068+0000 7f3231200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f322c0c5970 0x7f322c161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:20.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.068+0000 7f4658a00640 1 -- 172.21.15.138:0/4045529249 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46501576f0 con 0x7f46500c6620 2024-01-21T22:25:20.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.071+0000 7fcde9600640 1 -- 172.21.15.138:0/2872930786 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcddc07b0e0 con 0x7fcdec154e00 2024-01-21T22:25:20.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.072+0000 7fcde9600640 1 -- 172.21.15.138:0/2872930786 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fcddc076d20 con 0x7fcdec154e00 2024-01-21T22:25:20.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.074+0000 7f4aee000640 1 --2- 172.21.15.138:0/24667861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4af00c6620 0x7f4af00cc7c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4af006c780 tx=0x7f4ad802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a264c27b00a7dc52 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:20.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.076+0000 7f3231200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f322c0c5970 0x7f322c161e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:36280/0 (socket says 172.21.15.138:36280) 2024-01-21T22:25:20.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.076+0000 7f3231200640 1 -- 172.21.15.138:0/1910521246 learned_addr learned my addr 172.21.15.138:0/1910521246 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:20.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.085+0000 7f4aed600640 1 -- 172.21.15.138:0/24667861 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8030860 con 0x7f4af00c6620 2024-01-21T22:25:20.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.085+0000 7f4aed600640 1 -- 172.21.15.138:0/24667861 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad80309c0 con 0x7f4af00c6620 2024-01-21T22:25:20.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.085+0000 7f4aed600640 1 -- 172.21.15.138:0/24667861 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8030d50 con 0x7f4af00c6620 2024-01-21T22:25:20.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.086+0000 7f4af5a00640 1 -- 172.21.15.138:0/24667861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4af00c6620 msgr2=0x7f4af00cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.086+0000 7f4af5a00640 1 --2- 172.21.15.138:0/24667861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4af00c6620 0x7f4af00cc7c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4af006c780 tx=0x7f4ad802b9a0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.087+0000 7f4af5a00640 1 -- 172.21.15.138:0/24667861 shutdown_connections 2024-01-21T22:25:20.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.087+0000 7f4af5a00640 1 --2- 172.21.15.138:0/24667861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4af00c6620 0x7f4af00cc7c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.087+0000 7f4af5a00640 1 --2- 172.21.15.138:0/24667861 >> v2:172.21.15.152:3300/0 conn(0x7f4af00caf80 0x7f4af00ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.087+0000 7f4af5a00640 1 -- 172.21.15.138:0/24667861 >> 172.21.15.138:0/24667861 conn(0x7f4af00cbca0 msgr2=0x7f4af00cc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:20.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.087+0000 7f4af5a00640 1 -- 172.21.15.138:0/24667861 shutdown_connections 2024-01-21T22:25:20.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.087+0000 7f4af5a00640 1 -- 172.21.15.138:0/24667861 wait complete. 2024-01-21T22:25:20.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.087+0000 7f322b400640 1 --2- 172.21.15.138:0/1910521246 >> v2:172.21.15.138:3300/0 conn(0x7f322c06b9b0 0x7f322c158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:20.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.090+0000 7f4af5a00640 1 Processor -- start 2024-01-21T22:25:20.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.090+0000 7f4af5a00640 1 -- start start 2024-01-21T22:25:20.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.091+0000 7f4af5a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4af00c6620 0x7f4af00c4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:20.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.091+0000 7f4af5a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4af00caf80 0x7f4af014b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:20.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.091+0000 7f4af5a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4af0153ef0 0x7f4af0149b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:20.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.091+0000 7f4af5a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af00c4b40 con 0x7f4af00caf80 2024-01-21T22:25:20.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.091+0000 7f4af5a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af014a980 con 0x7f4af00c6620 2024-01-21T22:25:20.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.091+0000 7f4af5a00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af01f0910 con 0x7f4af0153ef0 2024-01-21T22:25:20.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.095+0000 7f322b400640 1 -- 172.21.15.138:0/1910521246 >> v2:172.21.15.152:3300/0 conn(0x7f322c0c5970 msgr2=0x7f322c161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:20.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.095+0000 7f322b400640 1 --2- 172.21.15.138:0/1910521246 >> v2:172.21.15.152:3300/0 conn(0x7f322c0c5970 0x7f322c161e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.095+0000 7f322b400640 1 -- 172.21.15.138:0/1910521246 >> v1:172.21.15.138:6789/0 conn(0x7f322c157ea0 legacy=0x7f322c06c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:20.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.095+0000 7f322b400640 1 -- 172.21.15.138:0/1910521246 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c15e5a0 con 0x7f322c06b9b0 2024-01-21T22:25:20.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.099+0000 7f322b400640 1 --2- 172.21.15.138:0/1910521246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f322c06b9b0 0x7f322c158280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f322c0c2400 tx=0x7f321402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41eafa0c4c1f415 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:20.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.106+0000 7f4aee000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4af00caf80 0x7f4af014b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:20.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.107+0000 7f322aa00640 1 -- 172.21.15.138:0/1910521246 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3214031830 con 0x7f322c06b9b0 2024-01-21T22:25:20.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.107+0000 7f322aa00640 1 -- 172.21.15.138:0/1910521246 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3214031990 con 0x7f322c06b9b0 2024-01-21T22:25:20.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.107+0000 7f322aa00640 1 -- 172.21.15.138:0/1910521246 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3214031d20 con 0x7f322c06b9b0 2024-01-21T22:25:20.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.107+0000 7f4aee000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4af00caf80 0x7f4af014b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53046/0 (socket says 172.21.15.138:53046) 2024-01-21T22:25:20.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.107+0000 7f4aee000640 1 -- 172.21.15.138:0/3323352606 learned_addr learned my addr 172.21.15.138:0/3323352606 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:20.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.108+0000 7f3233200640 1 -- 172.21.15.138:0/1910521246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f322c06b9b0 msgr2=0x7f322c158280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.108+0000 7f3233200640 1 --2- 172.21.15.138:0/1910521246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f322c06b9b0 0x7f322c158280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f322c0c2400 tx=0x7f321402b8d0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.108+0000 7f3233200640 1 -- 172.21.15.138:0/1910521246 shutdown_connections 2024-01-21T22:25:20.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.108+0000 7f3233200640 1 --2- 172.21.15.138:0/1910521246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f322c06b9b0 0x7f322c158280 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.108+0000 7f3233200640 1 --2- 172.21.15.138:0/1910521246 >> v2:172.21.15.152:3300/0 conn(0x7f322c0c5970 0x7f322c161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.108+0000 7f3233200640 1 -- 172.21.15.138:0/1910521246 >> 172.21.15.138:0/1910521246 conn(0x7f322c0cb070 msgr2=0x7f322c0cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:20.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.108+0000 7f3233200640 1 -- 172.21.15.138:0/1910521246 shutdown_connections 2024-01-21T22:25:20.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.108+0000 7f3233200640 1 -- 172.21.15.138:0/1910521246 wait complete. 2024-01-21T22:25:20.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.111+0000 7f3233200640 1 Processor -- start 2024-01-21T22:25:20.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.111+0000 7f3233200640 1 -- start start 2024-01-21T22:25:20.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.112+0000 7f3233200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f322c06b9b0 0x7f322c159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:20.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.112+0000 7f3233200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f322c0c5970 0x7f322c14cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:20.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.112+0000 7f3233200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f322c157ea0 0x7f322c152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:20.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.113+0000 7f3233200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c159840 con 0x7f322c157ea0 2024-01-21T22:25:20.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.113+0000 7f3233200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0ca2c0 con 0x7f322c06b9b0 2024-01-21T22:25:20.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.113+0000 7f3233200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c14d2f0 con 0x7f322c0c5970 2024-01-21T22:25:20.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.113+0000 7f4aee000640 1 -- 172.21.15.138:0/3323352606 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4af0153ef0 msgr2=0x7f4af0149b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.113+0000 7f4aee000640 1 --2- 172.21.15.138:0/3323352606 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4af0153ef0 0x7f4af0149b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.114+0000 7f4aee000640 1 -- 172.21.15.138:0/3323352606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4af00c6620 msgr2=0x7f4af00c4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.114+0000 7f4aee000640 1 --2- 172.21.15.138:0/3323352606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4af00c6620 0x7f4af00c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.114+0000 7f4aee000640 1 -- 172.21.15.138:0/3323352606 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad80094d0 con 0x7f4af00caf80 2024-01-21T22:25:20.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.119+0000 7f3231200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f322c157ea0 0x7f322c152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:20.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.119+0000 7f4aee000640 1 --2- 172.21.15.138:0/3323352606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4af00caf80 0x7f4af014b0b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8000c00 tx=0x7f4ad8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:20.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.120+0000 7f3231200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f322c157ea0 0x7f322c152700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53050/0 (socket says 172.21.15.138:53050) 2024-01-21T22:25:20.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.120+0000 7f3231200640 1 -- 172.21.15.138:0/189479184 learned_addr learned my addr 172.21.15.138:0/189479184 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:20.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.130+0000 7f3231200640 1 -- 172.21.15.138:0/189479184 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f322c0c5970 msgr2=0x7f322c14cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.130+0000 7f3231200640 1 --2- 172.21.15.138:0/189479184 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f322c0c5970 0x7f322c14cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.130+0000 7f3231200640 1 -- 172.21.15.138:0/189479184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f322c06b9b0 msgr2=0x7f322c159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.130+0000 7f3231200640 1 --2- 172.21.15.138:0/189479184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f322c06b9b0 0x7f322c159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.130+0000 7f3231200640 1 -- 172.21.15.138:0/189479184 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3214009400 con 0x7f322c157ea0 2024-01-21T22:25:20.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.132+0000 7f4ae6a00640 1 -- 172.21.15.138:0/3323352606 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8030830 con 0x7f4af00caf80 2024-01-21T22:25:20.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.133+0000 7f4ae6a00640 1 -- 172.21.15.138:0/3323352606 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad8043400 con 0x7f4af00caf80 2024-01-21T22:25:20.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.133+0000 7f4af5a00640 1 -- 172.21.15.138:0/3323352606 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af014ac20 con 0x7f4af00caf80 2024-01-21T22:25:20.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.133+0000 7f4ae6a00640 1 -- 172.21.15.138:0/3323352606 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad80437b0 con 0x7f4af00caf80 2024-01-21T22:25:20.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.133+0000 7f4af5a00640 1 -- 172.21.15.138:0/3323352606 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af01f27e0 con 0x7f4af00caf80 2024-01-21T22:25:20.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.134+0000 7f4af5a00640 1 -- 172.21.15.138:0/3323352606 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af01484a0 con 0x7f4af00caf80 2024-01-21T22:25:20.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.136+0000 7f3231200640 1 --2- 172.21.15.138:0/189479184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f322c157ea0 0x7f322c152700 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f322400efc0 tx=0x7f322400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:20.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.174+0000 7f4ae6a00640 1 -- 172.21.15.138:0/3323352606 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f4ad8043910 con 0x7f4af00caf80 2024-01-21T22:25:20.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.176+0000 7f4ae6a00640 1 --2- 172.21.15.138:0/3323352606 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f4ac806b380 0x7f4ac806d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:20.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.176+0000 7f4aeea00640 1 --2- 172.21.15.138:0/3323352606 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f4ac806b380 0x7f4ac806d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:20.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.177+0000 7f4aeea00640 1 --2- 172.21.15.138:0/3323352606 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f4ac806b380 0x7f4ac806d840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0006fd0 tx=0x7f4ae0006bc0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:20.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.181+0000 7f4ae6a00640 1 -- 172.21.15.138:0/3323352606 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ad803e070 con 0x7f4af00caf80 2024-01-21T22:25:20.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.244+0000 7f3223e00640 1 -- 172.21.15.138:0/189479184 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322400ccc0 con 0x7f322c157ea0 2024-01-21T22:25:20.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.244+0000 7f3233200640 1 -- 172.21.15.138:0/189479184 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322c1620f0 con 0x7f322c157ea0 2024-01-21T22:25:20.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.245+0000 7f3233200640 1 -- 172.21.15.138:0/189479184 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322c0c9d00 con 0x7f322c157ea0 2024-01-21T22:25:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.246+0000 7f3233200640 1 -- 172.21.15.138:0/189479184 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322c04ea30 con 0x7f322c157ea0 2024-01-21T22:25:20.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.266+0000 7f0753e00640 1 -- 172.21.15.138:0/2370767655 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f074c0d87d0 con 0x7f074c0d8fb0 2024-01-21T22:25:20.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.284+0000 7f7d0cc00640 1 -- 172.21.15.138:0/3599878148 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7ce0005810 con 0x7f7d180c0b20 2024-01-21T22:25:20.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.287+0000 7fcdf4400640 1 -- 172.21.15.138:0/2872930786 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fcdec0cb130 con 0x7fcdec154e00 2024-01-21T22:25:20.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.305+0000 7f83f2a00640 1 -- 172.21.15.138:0/2092804195 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f4003d80 con 0x7f83fc154040 2024-01-21T22:25:20.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.305+0000 7f83f2a00640 1 -- 172.21.15.138:0/2092804195 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f400ce90 con 0x7f83fc154040 2024-01-21T22:25:20.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.305+0000 7f83f2a00640 1 -- 172.21.15.138:0/2092804195 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f40052b0 con 0x7f83fc154040 2024-01-21T22:25:20.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.305+0000 7f8401a00640 1 -- 172.21.15.138:0/2092804195 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83fc0cd200 con 0x7f83fc154040 2024-01-21T22:25:20.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.306+0000 7f8401a00640 1 -- 172.21.15.138:0/2092804195 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83fc0cd770 con 0x7f83fc154040 2024-01-21T22:25:20.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.307+0000 7f8401a00640 1 -- 172.21.15.138:0/2092804195 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83fc0c9a10 con 0x7f83fc154040 2024-01-21T22:25:20.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.332+0000 7f464e000640 1 -- 172.21.15.138:0/4045529249 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f464001a400 con 0x7f46500c6620 2024-01-21T22:25:20.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.333+0000 7f38da000640 1 -- 172.21.15.138:0/1139384074 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f38cc010d30 con 0x7f38e40caf80 2024-01-21T22:25:20.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.334+0000 7f464e000640 1 --2- 172.21.15.138:0/4045529249 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f462806b380 0x7f462806d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:20.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.334+0000 7f4655600640 1 --2- 172.21.15.138:0/4045529249 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f462806b380 0x7f462806d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:20.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.334+0000 7f38da000640 1 --2- 172.21.15.138:0/1139384074 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f38bc06b330 0x7f38bc06d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:20.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.335+0000 7f38e1e00640 1 --2- 172.21.15.138:0/1139384074 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f38bc06b330 0x7f38bc06d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:20.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.335+0000 7f4655600640 1 --2- 172.21.15.138:0/4045529249 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f462806b380 0x7f462806d840 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4650069bb0 tx=0x7f463803b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:20.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.335+0000 7f38e1e00640 1 --2- 172.21.15.138:0/1139384074 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f38bc06b330 0x7f38bc06d7f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f38d4036000 tx=0x7f38d403b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:20.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.337+0000 7f38da000640 1 -- 172.21.15.138:0/1139384074 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38cc059050 con 0x7f38e40caf80 2024-01-21T22:25:20.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.338+0000 7f464e000640 1 -- 172.21.15.138:0/4045529249 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46400593b0 con 0x7f46500c6620 2024-01-21T22:25:20.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.349+0000 7f38da000640 1 -- 172.21.15.138:0/1139384074 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f38cc058720 con 0x7f38e40caf80 2024-01-21T22:25:20.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.350+0000 7f464e000640 1 -- 172.21.15.138:0/4045529249 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f4640054ff0 con 0x7f46500c6620 2024-01-21T22:25:20.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.368+0000 7f074aa00640 1 -- 172.21.15.138:0/2370767655 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f073c017090 con 0x7f074c0d8fb0 2024-01-21T22:25:20.371 INFO:teuthology.orchestra.run.smithi138.stdout:60129542155 2024-01-21T22:25:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.371+0000 7f0753e00640 1 -- 172.21.15.138:0/2370767655 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f072c06b380 msgr2=0x7f072c06d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.371+0000 7f0753e00640 1 --2- 172.21.15.138:0/2370767655 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f072c06b380 0x7f072c06d840 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0754050330 tx=0x7f07540690f0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.371+0000 7f0753e00640 1 -- 172.21.15.138:0/2370767655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f074c0d8fb0 msgr2=0x7f074c0d17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.371+0000 7f0753e00640 1 --2- 172.21.15.138:0/2370767655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f074c0d8fb0 0x7f074c0d17b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f073c00e870 tx=0x7f073c00ed30 comp rx=0 tx=0).stop 2024-01-21T22:25:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.372+0000 7f0753e00640 1 -- 172.21.15.138:0/2370767655 shutdown_connections 2024-01-21T22:25:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.372+0000 7f0753e00640 1 --2- 172.21.15.138:0/2370767655 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f072c06b380 0x7f072c06d840 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.372+0000 7f0753e00640 1 --2- 172.21.15.138:0/2370767655 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f074c0d8fb0 0x7f074c0d17b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.372+0000 7f0753e00640 1 --2- 172.21.15.138:0/2370767655 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f074c04c600 0x7f074c16e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.372+0000 7f0753e00640 1 --2- 172.21.15.138:0/2370767655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f074c046d70 0x7f074c0d3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.372+0000 7f0753e00640 1 -- 172.21.15.138:0/2370767655 >> 172.21.15.138:0/2370767655 conn(0x7f074c050180 msgr2=0x7f074c0343f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:20.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.372+0000 7f0753e00640 1 -- 172.21.15.138:0/2370767655 shutdown_connections 2024-01-21T22:25:20.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.372+0000 7f0753e00640 1 -- 172.21.15.138:0/2370767655 wait complete. 2024-01-21T22:25:20.396 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542155 for osd.3 2024-01-21T22:25:20.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.492+0000 7f4ae6a00640 1 -- 172.21.15.138:0/3323352606 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f4ad80786e0 con 0x7f4af00caf80 2024-01-21T22:25:20.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.530+0000 7fcde9600640 1 -- 172.21.15.138:0/2872930786 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcddc053030 con 0x7fcdec154e00 2024-01-21T22:25:20.533 INFO:teuthology.orchestra.run.smithi138.stdout:60129542154 2024-01-21T22:25:20.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.536+0000 7fcdf4400640 1 -- 172.21.15.138:0/2872930786 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcdc406b380 msgr2=0x7fcdc406d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.536+0000 7fcdf4400640 1 --2- 172.21.15.138:0/2872930786 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcdc406b380 0x7fcdc406d840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009fa0 tx=0x7fcde400b040 comp rx=0 tx=0).stop 2024-01-21T22:25:20.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.536+0000 7fcdf4400640 1 -- 172.21.15.138:0/2872930786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdec154e00 msgr2=0x7fcdec1f3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.536+0000 7fcdf4400640 1 --2- 172.21.15.138:0/2872930786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdec154e00 0x7fcdec1f3cc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcddc02b880 tx=0x7fcddc0046c0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.536+0000 7fcdf4400640 1 -- 172.21.15.138:0/2872930786 shutdown_connections 2024-01-21T22:25:20.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.536+0000 7fcdf4400640 1 --2- 172.21.15.138:0/2872930786 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcdc406b380 0x7fcdc406d840 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.536+0000 7fcdf4400640 1 --2- 172.21.15.138:0/2872930786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdec154e00 0x7fcdec1f3cc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.536+0000 7fcdf4400640 1 --2- 172.21.15.138:0/2872930786 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcdec0cc7c0 0x7fcdec0c5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.536+0000 7fcdf4400640 1 --2- 172.21.15.138:0/2872930786 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcdec06bd40 0x7fcdec0c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.536+0000 7fcdf4400640 1 -- 172.21.15.138:0/2872930786 >> 172.21.15.138:0/2872930786 conn(0x7fcdec0cbca0 msgr2=0x7fcdec06a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:20.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.536+0000 7fcdf4400640 1 -- 172.21.15.138:0/2872930786 shutdown_connections 2024-01-21T22:25:20.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.537+0000 7fcdf4400640 1 -- 172.21.15.138:0/2872930786 wait complete. 2024-01-21T22:25:20.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.543+0000 7f38e8e00640 1 -- 172.21.15.138:0/1139384074 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f38e40b4e50 con 0x7f38e40caf80 2024-01-21T22:25:20.563 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.4 2024-01-21T22:25:20.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.576+0000 7f4658a00640 1 -- 172.21.15.138:0/4045529249 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4650152250 con 0x7f46500c6620 2024-01-21T22:25:20.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.580+0000 7f464e000640 1 -- 172.21.15.138:0/4045529249 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4640015070 con 0x7f46500c6620 2024-01-21T22:25:20.582 INFO:teuthology.orchestra.run.smithi138.stdout:60129542154 2024-01-21T22:25:20.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.583+0000 7f4658a00640 1 -- 172.21.15.138:0/4045529249 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f462806b380 msgr2=0x7f462806d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.583+0000 7f4658a00640 1 --2- 172.21.15.138:0/4045529249 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f462806b380 0x7f462806d840 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4650069bb0 tx=0x7f463803b040 comp rx=0 tx=0).stop 2024-01-21T22:25:20.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.583+0000 7f4658a00640 1 -- 172.21.15.138:0/4045529249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46500c6620 msgr2=0x7f46500c3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.583+0000 7f4658a00640 1 --2- 172.21.15.138:0/4045529249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46500c6620 0x7f46500c3ec0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f464000b410 tx=0x7f464000b8d0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.583+0000 7f4658a00640 1 -- 172.21.15.138:0/4045529249 shutdown_connections 2024-01-21T22:25:20.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.583+0000 7f4658a00640 1 --2- 172.21.15.138:0/4045529249 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f462806b380 0x7f462806d840 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.583+0000 7f4658a00640 1 --2- 172.21.15.138:0/4045529249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4650153ef0 0x7f46500c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.583+0000 7f4658a00640 1 --2- 172.21.15.138:0/4045529249 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f46500caf80 0x7f46500c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.583+0000 7f4658a00640 1 --2- 172.21.15.138:0/4045529249 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f46500c6620 0x7f46500c3ec0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.583+0000 7f4658a00640 1 -- 172.21.15.138:0/4045529249 >> 172.21.15.138:0/4045529249 conn(0x7f46500cbca0 msgr2=0x7f4650159380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:20.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.583+0000 7f4658a00640 1 -- 172.21.15.138:0/4045529249 shutdown_connections 2024-01-21T22:25:20.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.583+0000 7f4658a00640 1 -- 172.21.15.138:0/4045529249 wait complete. 2024-01-21T22:25:20.600 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.1 2024-01-21T22:25:20.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.604+0000 7f83f2a00640 1 -- 172.21.15.138:0/2092804195 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f83f4005410 con 0x7f83fc154040 2024-01-21T22:25:20.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.605+0000 7f83f2a00640 1 --2- 172.21.15.138:0/2092804195 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f83d406b3b0 0x7f83d406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:20.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.605+0000 7f83faa00640 1 --2- 172.21.15.138:0/2092804195 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f83d406b3b0 0x7f83d406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:20.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.606+0000 7f83faa00640 1 --2- 172.21.15.138:0/2092804195 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f83d406b3b0 0x7f83d406d870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009710 tx=0x7f83ec006c90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:20.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.606+0000 7f83f2a00640 1 -- 172.21.15.138:0/2092804195 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83f4058910 con 0x7f83fc154040 2024-01-21T22:25:20.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.619+0000 7f83f2a00640 1 -- 172.21.15.138:0/2092804195 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f83f4054550 con 0x7f83fc154040 2024-01-21T22:25:20.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.624+0000 7f7d0f400640 1 -- 172.21.15.138:0/3599878148 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7d10019070 con 0x7f7d180c0b20 2024-01-21T22:25:20.627 INFO:teuthology.orchestra.run.smithi138.stdout:60129542155 2024-01-21T22:25:20.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.626+0000 7f38da000640 1 -- 172.21.15.138:0/1139384074 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f38cc01e020 con 0x7f38e40caf80 2024-01-21T22:25:20.628 INFO:teuthology.orchestra.run.smithi138.stdout:60129542154 2024-01-21T22:25:20.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.628+0000 7f7d0cc00640 1 -- 172.21.15.138:0/3599878148 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7cf006b3c0 msgr2=0x7f7cf006d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.628+0000 7f7d0cc00640 1 --2- 172.21.15.138:0/3599878148 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7cf006b3c0 0x7f7cf006d880 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7d08039000 tx=0x7f7d08008040 comp rx=0 tx=0).stop 2024-01-21T22:25:20.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.628+0000 7f7d0cc00640 1 -- 172.21.15.138:0/3599878148 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d180c0b20 msgr2=0x7f7d18157aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.628+0000 7f7d0cc00640 1 --2- 172.21.15.138:0/3599878148 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d180c0b20 0x7f7d18157aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000ba50 tx=0x7f7d1000bf10 comp rx=0 tx=0).stop 2024-01-21T22:25:20.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.628+0000 7f7d0cc00640 1 -- 172.21.15.138:0/3599878148 shutdown_connections 2024-01-21T22:25:20.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.628+0000 7f7d0cc00640 1 --2- 172.21.15.138:0/3599878148 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7cf006b3c0 0x7f7cf006d880 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.628+0000 7f7d0cc00640 1 --2- 172.21.15.138:0/3599878148 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7d18152f20 0x7f7d181eb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.628+0000 7f7d0cc00640 1 --2- 172.21.15.138:0/3599878148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d180c7540 0x7f7d180723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.628+0000 7f7d0cc00640 1 --2- 172.21.15.138:0/3599878148 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7d180c0b20 0x7f7d18157aa0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.628+0000 7f7d0cc00640 1 -- 172.21.15.138:0/3599878148 >> 172.21.15.138:0/3599878148 conn(0x7f7d1806d930 msgr2=0x7f7d180706f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:20.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.628+0000 7f7d0cc00640 1 -- 172.21.15.138:0/3599878148 shutdown_connections 2024-01-21T22:25:20.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.628+0000 7f7d0cc00640 1 -- 172.21.15.138:0/3599878148 wait complete. 2024-01-21T22:25:20.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.629+0000 7f38e8e00640 1 -- 172.21.15.138:0/1139384074 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f38bc06b330 msgr2=0x7f38bc06d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.629+0000 7f38e8e00640 1 --2- 172.21.15.138:0/1139384074 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f38bc06b330 0x7f38bc06d7f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f38d4036000 tx=0x7f38d403b040 comp rx=0 tx=0).stop 2024-01-21T22:25:20.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.629+0000 7f38e8e00640 1 -- 172.21.15.138:0/1139384074 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38e40caf80 msgr2=0x7f38e40c3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.629+0000 7f38e8e00640 1 --2- 172.21.15.138:0/1139384074 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38e40caf80 0x7f38e40c3460 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00b4d0 tx=0x7f38cc00b990 comp rx=0 tx=0).stop 2024-01-21T22:25:20.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.629+0000 7f38e8e00640 1 -- 172.21.15.138:0/1139384074 shutdown_connections 2024-01-21T22:25:20.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.629+0000 7f38e8e00640 1 --2- 172.21.15.138:0/1139384074 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f38bc06b330 0x7f38bc06d7f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.629+0000 7f38e8e00640 1 --2- 172.21.15.138:0/1139384074 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f38e4153ef0 0x7f38e4146de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.629+0000 7f38e8e00640 1 --2- 172.21.15.138:0/1139384074 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f38e40caf80 0x7f38e40c3460 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.629+0000 7f38e8e00640 1 --2- 172.21.15.138:0/1139384074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38e40c6620 0x7f38e40c2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.629+0000 7f38e8e00640 1 -- 172.21.15.138:0/1139384074 >> 172.21.15.138:0/1139384074 conn(0x7f38e40cbca0 msgr2=0x7f38e406a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:20.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.629+0000 7f38e8e00640 1 -- 172.21.15.138:0/1139384074 shutdown_connections 2024-01-21T22:25:20.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.629+0000 7f38e8e00640 1 -- 172.21.15.138:0/1139384074 wait complete. 2024-01-21T22:25:20.649 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.6 2024-01-21T22:25:20.656 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542155 for osd.0 2024-01-21T22:25:20.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.729+0000 7f4af5a00640 1 -- 172.21.15.138:0/3323352606 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4af004ea30 con 0x7f4af00caf80 2024-01-21T22:25:20.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.808+0000 7f3223e00640 1 -- 172.21.15.138:0/189479184 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322400f040 con 0x7f322c157ea0 2024-01-21T22:25:20.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.808+0000 7f3223e00640 1 -- 172.21.15.138:0/189479184 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224003bb0 con 0x7f322c157ea0 2024-01-21T22:25:20.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.810+0000 7f3223e00640 1 -- 172.21.15.138:0/189479184 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f3224003d50 con 0x7f322c157ea0 2024-01-21T22:25:20.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.811+0000 7f3223e00640 1 --2- 172.21.15.138:0/189479184 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f320406b330 0x7f320406d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:20.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.812+0000 7f322be00640 1 --2- 172.21.15.138:0/189479184 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f320406b330 0x7f320406d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:20.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.813+0000 7f322be00640 1 --2- 172.21.15.138:0/189479184 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f320406b330 0x7f320406d7f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f321c009650 tx=0x7f321c006ba0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:20.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.823+0000 7f8401a00640 1 -- 172.21.15.138:0/2092804195 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f83fc04ea30 con 0x7f83fc154040 2024-01-21T22:25:20.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.826+0000 7f83f2a00640 1 -- 172.21.15.138:0/2092804195 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f83f4017090 con 0x7f83fc154040 2024-01-21T22:25:20.828 INFO:teuthology.orchestra.run.smithi138.stdout:60129542153 2024-01-21T22:25:20.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.828+0000 7f8401a00640 1 -- 172.21.15.138:0/2092804195 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f83d406b3b0 msgr2=0x7f83d406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.828+0000 7f8401a00640 1 --2- 172.21.15.138:0/2092804195 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f83d406b3b0 0x7f83d406d870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009710 tx=0x7f83ec006c90 comp rx=0 tx=0).stop 2024-01-21T22:25:20.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.828+0000 7f8401a00640 1 -- 172.21.15.138:0/2092804195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83fc154040 msgr2=0x7f83fc0c66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.828+0000 7f8401a00640 1 --2- 172.21.15.138:0/2092804195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83fc154040 0x7f83fc0c66a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f83f400c780 tx=0x7f83f400cc40 comp rx=0 tx=0).stop 2024-01-21T22:25:20.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.829+0000 7f8401a00640 1 -- 172.21.15.138:0/2092804195 shutdown_connections 2024-01-21T22:25:20.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.829+0000 7f8401a00640 1 --2- 172.21.15.138:0/2092804195 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f83d406b3b0 0x7f83d406d870 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.829+0000 7f8401a00640 1 --2- 172.21.15.138:0/2092804195 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f83fc154040 0x7f83fc0c66a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.829+0000 7f8401a00640 1 --2- 172.21.15.138:0/2092804195 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f83fc153c60 0x7f83fc0ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.829+0000 7f8401a00640 1 --2- 172.21.15.138:0/2092804195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83fc0c4500 0x7f83fc1f7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.829+0000 7f8401a00640 1 -- 172.21.15.138:0/2092804195 >> 172.21.15.138:0/2092804195 conn(0x7f83fc06b630 msgr2=0x7f83fc0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:20.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.829+0000 7f8401a00640 1 -- 172.21.15.138:0/2092804195 shutdown_connections 2024-01-21T22:25:20.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.829+0000 7f8401a00640 1 -- 172.21.15.138:0/2092804195 wait complete. 2024-01-21T22:25:20.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.840+0000 7f3223e00640 1 -- 172.21.15.138:0/189479184 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f322400ce20 con 0x7f322c157ea0 2024-01-21T22:25:20.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.841+0000 7f3223e00640 1 -- 172.21.15.138:0/189479184 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f3224055c60 con 0x7f322c157ea0 2024-01-21T22:25:20.850 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.2 2024-01-21T22:25:20.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.868+0000 7f4ae6a00640 1 -- 172.21.15.138:0/3323352606 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4ad8034020 con 0x7f4af00caf80 2024-01-21T22:25:20.871 INFO:teuthology.orchestra.run.smithi138.stdout:60129542154 2024-01-21T22:25:20.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.872+0000 7f4af5a00640 1 -- 172.21.15.138:0/3323352606 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f4ac806b380 msgr2=0x7f4ac806d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.872+0000 7f4af5a00640 1 --2- 172.21.15.138:0/3323352606 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f4ac806b380 0x7f4ac806d840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0006fd0 tx=0x7f4ae0006bc0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.872+0000 7f4af5a00640 1 -- 172.21.15.138:0/3323352606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4af00caf80 msgr2=0x7f4af014b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:20.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.872+0000 7f4af5a00640 1 --2- 172.21.15.138:0/3323352606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4af00caf80 0x7f4af014b0b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8000c00 tx=0x7f4ad8004730 comp rx=0 tx=0).stop 2024-01-21T22:25:20.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.872+0000 7f4af5a00640 1 -- 172.21.15.138:0/3323352606 shutdown_connections 2024-01-21T22:25:20.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.872+0000 7f4af5a00640 1 --2- 172.21.15.138:0/3323352606 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f4ac806b380 0x7f4ac806d840 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.872+0000 7f4af5a00640 1 --2- 172.21.15.138:0/3323352606 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4af0153ef0 0x7f4af0149b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.872+0000 7f4af5a00640 1 --2- 172.21.15.138:0/3323352606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4af00caf80 0x7f4af014b0b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.872+0000 7f4af5a00640 1 --2- 172.21.15.138:0/3323352606 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4af00c6620 0x7f4af00c4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:20.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.872+0000 7f4af5a00640 1 -- 172.21.15.138:0/3323352606 >> 172.21.15.138:0/3323352606 conn(0x7f4af00cbca0 msgr2=0x7f4af00c8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:20.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.872+0000 7f4af5a00640 1 -- 172.21.15.138:0/3323352606 shutdown_connections 2024-01-21T22:25:20.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:20.872+0000 7f4af5a00640 1 -- 172.21.15.138:0/3323352606 wait complete. 2024-01-21T22:25:20.894 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.5 2024-01-21T22:25:21.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.049+0000 7f3233200640 1 -- 172.21.15.138:0/189479184 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f322c151160 con 0x7f322c157ea0 2024-01-21T22:25:21.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.123+0000 7f3223e00640 1 -- 172.21.15.138:0/189479184 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3224059280 con 0x7f322c157ea0 2024-01-21T22:25:21.126 INFO:teuthology.orchestra.run.smithi138.stdout:60129542154 2024-01-21T22:25:21.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.126+0000 7f3233200640 1 -- 172.21.15.138:0/189479184 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f320406b330 msgr2=0x7f320406d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:21.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.126+0000 7f3233200640 1 --2- 172.21.15.138:0/189479184 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f320406b330 0x7f320406d7f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f321c009650 tx=0x7f321c006ba0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.126+0000 7f3233200640 1 -- 172.21.15.138:0/189479184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f322c157ea0 msgr2=0x7f322c152700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:21.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.126+0000 7f3233200640 1 --2- 172.21.15.138:0/189479184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f322c157ea0 0x7f322c152700 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f322400efc0 tx=0x7f322400c490 comp rx=0 tx=0).stop 2024-01-21T22:25:21.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.127+0000 7f3233200640 1 -- 172.21.15.138:0/189479184 shutdown_connections 2024-01-21T22:25:21.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.127+0000 7f3233200640 1 --2- 172.21.15.138:0/189479184 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f320406b330 0x7f320406d7f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.127+0000 7f3233200640 1 --2- 172.21.15.138:0/189479184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f322c157ea0 0x7f322c152700 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.127+0000 7f3233200640 1 --2- 172.21.15.138:0/189479184 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f322c0c5970 0x7f322c14cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.127+0000 7f3233200640 1 --2- 172.21.15.138:0/189479184 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f322c06b9b0 0x7f322c159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.127+0000 7f3233200640 1 -- 172.21.15.138:0/189479184 >> 172.21.15.138:0/189479184 conn(0x7f322c0cb070 msgr2=0x7f322c15bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:21.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.127+0000 7f3233200640 1 -- 172.21.15.138:0/189479184 shutdown_connections 2024-01-21T22:25:21.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.127+0000 7f3233200640 1 -- 172.21.15.138:0/189479184 wait complete. 2024-01-21T22:25:21.146 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.7 2024-01-21T22:25:21.397 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-21T22:25:21.564 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-01-21T22:25:21.601 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-21T22:25:21.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.607+0000 7fcd14a00640 1 Processor -- start 2024-01-21T22:25:21.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.608+0000 7fcd14a00640 1 -- start start 2024-01-21T22:25:21.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.609+0000 7fcd14a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcd0c0cc940 0x7fcd0c0ca710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.609+0000 7fcd14a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd0c154100 con 0x7fcd0c153d20 2024-01-21T22:25:21.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.609+0000 7fcd14a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd0c0c4810 con 0x7fcd0c0c43e0 2024-01-21T22:25:21.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.609+0000 7fcd14a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd0c0ca500 con 0x7fcd0c0cc940 2024-01-21T22:25:21.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.610+0000 7fcd11600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fcd0c0cc940 0x7fcd0c0ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:21.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.610+0000 7fcd12000640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fcd0c153d20 0x7fcd0c15a3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:37960/0 (socket says 172.21.15.138:37960) 2024-01-21T22:25:21.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.610+0000 7fcd12000640 1 -- 172.21.15.138:0/445118608 learned_addr learned my addr 172.21.15.138:0/445118608 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:21.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.613+0000 7fcd11600640 1 -- 172.21.15.138:0/445118608 >> v1:172.21.15.152:6789/0 conn(0x7fcd0c0c43e0 legacy=0x7fcd0c0cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:21.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.613+0000 7fcd11600640 1 -- 172.21.15.138:0/445118608 >> v1:172.21.15.138:6789/0 conn(0x7fcd0c153d20 legacy=0x7fcd0c15a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:21.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.613+0000 7fcd11600640 1 -- 172.21.15.138:0/445118608 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c0cb5c0 con 0x7fcd0c0cc940 2024-01-21T22:25:21.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.616+0000 7fcd11600640 1 --2- 172.21.15.138:0/445118608 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd0c0cc940 0x7fcd0c0ca710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fccf40097a0 tx=0x7fccf402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf928a3d5d1c3ff9 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:21.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.620+0000 7fcd10c00640 1 -- 172.21.15.138:0/445118608 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf4030610 con 0x7fcd0c0cc940 2024-01-21T22:25:21.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.620+0000 7fcd10c00640 1 -- 172.21.15.138:0/445118608 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf4030770 con 0x7fcd0c0cc940 2024-01-21T22:25:21.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.620+0000 7fcd10c00640 1 -- 172.21.15.138:0/445118608 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf4030b00 con 0x7fcd0c0cc940 2024-01-21T22:25:21.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.620+0000 7fcd14a00640 1 -- 172.21.15.138:0/445118608 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd0c0cc940 msgr2=0x7fcd0c0ca710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:21.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.620+0000 7fcd14a00640 1 --2- 172.21.15.138:0/445118608 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd0c0cc940 0x7fcd0c0ca710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fccf40097a0 tx=0x7fccf402b960 comp rx=0 tx=0).stop 2024-01-21T22:25:21.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.621+0000 7fcd14a00640 1 -- 172.21.15.138:0/445118608 shutdown_connections 2024-01-21T22:25:21.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.621+0000 7fcd14a00640 1 --2- 172.21.15.138:0/445118608 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd0c0cc940 0x7fcd0c0ca710 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.621+0000 7fcd14a00640 1 -- 172.21.15.138:0/445118608 >> 172.21.15.138:0/445118608 conn(0x7fcd0c06d0d0 msgr2=0x7fcd0c06af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:21.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.621+0000 7fcd14a00640 1 -- 172.21.15.138:0/445118608 shutdown_connections 2024-01-21T22:25:21.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.621+0000 7fcd14a00640 1 -- 172.21.15.138:0/445118608 wait complete. 2024-01-21T22:25:21.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.622+0000 7fcd14a00640 1 Processor -- start 2024-01-21T22:25:21.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.623+0000 7fcd14a00640 1 -- start start 2024-01-21T22:25:21.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.623+0000 7fcd14a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd0c0c43e0 0x7fcd0c1f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.623+0000 7fcd14a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd0c0cc940 0x7fcd0c1521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.623+0000 7fcd14a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcd0c153d20 0x7fcd0c1f1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.623+0000 7fcd14a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c0cf280 con 0x7fcd0c0cc940 2024-01-21T22:25:21.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.623+0000 7fcd14a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c158510 con 0x7fcd0c0c43e0 2024-01-21T22:25:21.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.623+0000 7fcd14a00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0c1597c0 con 0x7fcd0c153d20 2024-01-21T22:25:21.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.624+0000 7fcd12000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd0c0c43e0 0x7fcd0c1f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:21.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.624+0000 7fcd12a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcd0c153d20 0x7fcd0c1f1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:21.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.624+0000 7fcd12000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd0c0c43e0 0x7fcd0c1f3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:36320/0 (socket says 172.21.15.138:36320) 2024-01-21T22:25:21.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.624+0000 7fcd12000640 1 -- 172.21.15.138:0/3535120938 learned_addr learned my addr 172.21.15.138:0/3535120938 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:21.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.624+0000 7fcd11600640 1 --2- 172.21.15.138:0/3535120938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd0c0cc940 0x7fcd0c1521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:21.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.628+0000 7fcd12000640 1 -- 172.21.15.138:0/3535120938 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcd0c153d20 msgr2=0x7fcd0c1f1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:21.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.628+0000 7fcd12000640 1 --2- 172.21.15.138:0/3535120938 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcd0c153d20 0x7fcd0c1f1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.628+0000 7fcd12000640 1 -- 172.21.15.138:0/3535120938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd0c0cc940 msgr2=0x7fcd0c1521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:21.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.628+0000 7fcd12000640 1 --2- 172.21.15.138:0/3535120938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd0c0cc940 0x7fcd0c1521a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.628+0000 7fcd12000640 1 -- 172.21.15.138:0/3535120938 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf4009400 con 0x7fcd0c0c43e0 2024-01-21T22:25:21.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.630+0000 7fcd12000640 1 --2- 172.21.15.138:0/3535120938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd0c0c43e0 0x7fcd0c1f3af0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00c3f0 tx=0x7fccfc00c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:21.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.633+0000 7fcd0a000640 1 -- 172.21.15.138:0/3535120938 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc005ce0 con 0x7fcd0c0c43e0 2024-01-21T22:25:21.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.634+0000 7fcd0a000640 1 -- 172.21.15.138:0/3535120938 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccfc005e40 con 0x7fcd0c0c43e0 2024-01-21T22:25:21.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.634+0000 7fcd0a000640 1 -- 172.21.15.138:0/3535120938 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc0072f0 con 0x7fcd0c0c43e0 2024-01-21T22:25:21.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.634+0000 7fcd14a00640 1 -- 172.21.15.138:0/3535120938 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd0c1f4030 con 0x7fcd0c0c43e0 2024-01-21T22:25:21.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.634+0000 7fcd14a00640 1 -- 172.21.15.138:0/3535120938 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd0c157750 con 0x7fcd0c0c43e0 2024-01-21T22:25:21.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.635+0000 7fcd14a00640 1 -- 172.21.15.138:0/3535120938 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd0c04ea30 con 0x7fcd0c0c43e0 2024-01-21T22:25:21.649 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-01-21T22:25:21.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.653+0000 7fcd0a000640 1 -- 172.21.15.138:0/3535120938 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fccfc007450 con 0x7fcd0c0c43e0 2024-01-21T22:25:21.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.654+0000 7fcd0a000640 1 --2- 172.21.15.138:0/3535120938 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcce406b380 0x7fcce406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.655+0000 7fcd11600640 1 --2- 172.21.15.138:0/3535120938 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcce406b380 0x7fcce406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:21.657 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-21T22:25:21.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.655+0000 7fcd11600640 1 --2- 172.21.15.138:0/3535120938 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcce406b380 0x7fcce406d840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fccf4036000 tx=0x7fccf4008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:21.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.758+0000 7f5f31600640 1 Processor -- start 2024-01-21T22:25:21.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.759+0000 7f5f31600640 1 -- start start 2024-01-21T22:25:21.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.760+0000 7f5f31600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5f2c06c5d0 0x7f5f2c0c6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.760+0000 7f5f31600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f2c06c9b0 0x7f5f2c0c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.760+0000 7f5f31600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5f2c0c5970 0x7f5f2c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.761+0000 7f5f31600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f2c0c3d30 con 0x7f5f2c06c5d0 2024-01-21T22:25:21.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.761+0000 7f5f31600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f2c0c55a0 con 0x7f5f2c0c5970 2024-01-21T22:25:21.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.761+0000 7f5f31600640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0c4310 con 0x7f5f2c06c9b0 2024-01-21T22:25:21.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.761+0000 7f5f2a000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f2c06c9b0 0x7f5f2c0c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:21.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.761+0000 7f5f2aa00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5f2c06c5d0 0x7f5f2c0c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:21.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.762+0000 7f5f2a000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f2c06c9b0 0x7f5f2c0c5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42486/0 (socket says 172.21.15.138:42486) 2024-01-21T22:25:21.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.762+0000 7f5f2a000640 1 -- 172.21.15.138:0/3299729779 learned_addr learned my addr 172.21.15.138:0/3299729779 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:21.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.764+0000 7f5f2a000640 1 -- 172.21.15.138:0/3299729779 >> v2:172.21.15.152:3300/0 conn(0x7f5f2c0c5970 msgr2=0x7f5f2c06d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:21.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.764+0000 7f5f2a000640 1 --2- 172.21.15.138:0/3299729779 >> v2:172.21.15.152:3300/0 conn(0x7f5f2c0c5970 0x7f5f2c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.764+0000 7f5f2a000640 1 -- 172.21.15.138:0/3299729779 >> v2:172.21.15.138:3300/0 conn(0x7f5f2c06c5d0 msgr2=0x7f5f2c0c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:21.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.764+0000 7f5f2a000640 1 --2- 172.21.15.138:0/3299729779 >> v2:172.21.15.138:3300/0 conn(0x7f5f2c06c5d0 0x7f5f2c0c6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.764+0000 7f5f2a000640 1 -- 172.21.15.138:0/3299729779 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c0c28d0 con 0x7f5f2c06c9b0 2024-01-21T22:25:21.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.766+0000 7f5f2a000640 1 --2- 172.21.15.138:0/3299729779 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f2c06c9b0 0x7f5f2c0c5060 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c0c5f00 tx=0x7f5f1402b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d41e0f5cb299ad5 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:21.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.770+0000 7f5f29600640 1 -- 172.21.15.138:0/3299729779 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f14030860 con 0x7f5f2c06c9b0 2024-01-21T22:25:21.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.770+0000 7f5f29600640 1 -- 172.21.15.138:0/3299729779 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f140309c0 con 0x7f5f2c06c9b0 2024-01-21T22:25:21.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.770+0000 7f5f29600640 1 -- 172.21.15.138:0/3299729779 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f14030d50 con 0x7f5f2c06c9b0 2024-01-21T22:25:21.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.771+0000 7f5f31600640 1 -- 172.21.15.138:0/3299729779 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f2c06c9b0 msgr2=0x7f5f2c0c5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:21.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.771+0000 7f5f31600640 1 --2- 172.21.15.138:0/3299729779 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f2c06c9b0 0x7f5f2c0c5060 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c0c5f00 tx=0x7f5f1402b9a0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.771+0000 7f5f31600640 1 -- 172.21.15.138:0/3299729779 shutdown_connections 2024-01-21T22:25:21.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.771+0000 7f5f31600640 1 --2- 172.21.15.138:0/3299729779 >> v2:172.21.15.152:3300/0 conn(0x7f5f2c0c5970 0x7f5f2c06d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.771+0000 7f5f31600640 1 --2- 172.21.15.138:0/3299729779 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f2c06c9b0 0x7f5f2c0c5060 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.771+0000 7f5f31600640 1 --2- 172.21.15.138:0/3299729779 >> v2:172.21.15.138:3300/0 conn(0x7f5f2c06c5d0 0x7f5f2c0c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.771+0000 7f5f31600640 1 -- 172.21.15.138:0/3299729779 >> 172.21.15.138:0/3299729779 conn(0x7f5f2c0cb070 msgr2=0x7f5f2c0cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:21.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.772+0000 7f5f31600640 1 -- 172.21.15.138:0/3299729779 shutdown_connections 2024-01-21T22:25:21.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.772+0000 7f5f31600640 1 -- 172.21.15.138:0/3299729779 wait complete. 2024-01-21T22:25:21.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.774+0000 7f5f31600640 1 Processor -- start 2024-01-21T22:25:21.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.774+0000 7f5f31600640 1 -- start start 2024-01-21T22:25:21.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.776+0000 7f5f31600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f2c06c5d0 0x7f5f2c0af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.776+0000 7f5f31600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f2c06c9b0 0x7f5f2c1f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.776+0000 7f5f31600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f2c0c5970 0x7f5f2c0715c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.776+0000 7f5f31600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c1ef430 con 0x7f5f2c06c9b0 2024-01-21T22:25:21.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.776+0000 7f5f31600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c155150 con 0x7f5f2c0c5970 2024-01-21T22:25:21.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.776+0000 7f5f31600640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c155430 con 0x7f5f2c06c5d0 2024-01-21T22:25:21.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.776+0000 7f5f2aa00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f2c06c5d0 0x7f5f2c0af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:21.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.777+0000 7f5f2aa00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f2c06c5d0 0x7f5f2c0af8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42500/0 (socket says 172.21.15.138:42500) 2024-01-21T22:25:21.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.777+0000 7f5f2aa00640 1 -- 172.21.15.138:0/1038868746 learned_addr learned my addr 172.21.15.138:0/1038868746 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:21.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.777+0000 7f5f2a000640 1 --2- 172.21.15.138:0/1038868746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f2c06c9b0 0x7f5f2c1f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:21.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.780+0000 7f5f2aa00640 1 -- 172.21.15.138:0/1038868746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f2c0c5970 msgr2=0x7f5f2c0715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:21.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.780+0000 7f5f2aa00640 1 --2- 172.21.15.138:0/1038868746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f2c0c5970 0x7f5f2c0715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.780+0000 7f5f2aa00640 1 -- 172.21.15.138:0/1038868746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f2c06c9b0 msgr2=0x7f5f2c1f2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:21.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.780+0000 7f5f2aa00640 1 --2- 172.21.15.138:0/1038868746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f2c06c9b0 0x7f5f2c1f2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.780+0000 7f5f2aa00640 1 -- 172.21.15.138:0/1038868746 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f140094d0 con 0x7f5f2c06c5d0 2024-01-21T22:25:21.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.782+0000 7f5f2aa00640 1 --2- 172.21.15.138:0/1038868746 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f2c06c5d0 0x7f5f2c0af8c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00e9b0 tx=0x7f5f1c00ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:21.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.786+0000 7f5f22a00640 1 -- 172.21.15.138:0/1038868746 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c009670 con 0x7f5f2c06c5d0 2024-01-21T22:25:21.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.786+0000 7f5f22a00640 1 -- 172.21.15.138:0/1038868746 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c00c9a0 con 0x7f5f2c06c5d0 2024-01-21T22:25:21.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.786+0000 7f5f31600640 1 -- 172.21.15.138:0/1038868746 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2c1ef6f0 con 0x7f5f2c06c5d0 2024-01-21T22:25:21.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.786+0000 7f4f6c800640 1 Processor -- start 2024-01-21T22:25:21.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.786+0000 7f4f6c800640 1 -- start start 2024-01-21T22:25:21.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.786+0000 7f5f22a00640 1 -- 172.21.15.138:0/1038868746 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c00cd50 con 0x7f5f2c06c5d0 2024-01-21T22:25:21.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.786+0000 7f5f31600640 1 -- 172.21.15.138:0/1038868746 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2c1efcb0 con 0x7f5f2c06c5d0 2024-01-21T22:25:21.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.787+0000 7f5f31600640 1 -- 172.21.15.138:0/1038868746 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2c069550 con 0x7f5f2c06c5d0 2024-01-21T22:25:21.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.787+0000 7f4f6c800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f4f64157ea0 0x7f4f64158280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.787+0000 7f4f6c800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f640c5970 0x7f4f64161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.787+0000 7f4f6c800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6406be80 con 0x7f4f6406b9b0 2024-01-21T22:25:21.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.787+0000 7f4f6c800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f6415dfe0 con 0x7f4f64157ea0 2024-01-21T22:25:21.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.787+0000 7f4f6c800640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6415e2c0 con 0x7f4f640c5970 2024-01-21T22:25:21.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.788+0000 7f4f69400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f4f6406b9b0 0x7f4f6406c360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:37964/0 (socket says 172.21.15.138:37964) 2024-01-21T22:25:21.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.788+0000 7f4f69400640 1 -- 172.21.15.138:0/2889939906 learned_addr learned my addr 172.21.15.138:0/2889939906 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:21.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.791+0000 7f4f68a00640 1 -- 172.21.15.138:0/2889939906 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 231270306 0 0) 0x7f4f6406be80 con 0x7f4f6406b9b0 2024-01-21T22:25:21.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.791+0000 7f4f68a00640 1 -- 172.21.15.138:0/2889939906 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f44003520 con 0x7f4f6406b9b0 2024-01-21T22:25:21.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.795+0000 7f4f68a00640 1 -- 172.21.15.138:0/2889939906 <== mon.0 v1:172.21.15.138:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1370946647 0 0) 0x7f4f44003520 con 0x7f4f6406b9b0 2024-01-21T22:25:21.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.795+0000 7f4f68a00640 1 -- 172.21.15.138:0/2889939906 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f640c5970 msgr2=0x7f4f64161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:21.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.795+0000 7f4f68a00640 1 --2- 172.21.15.138:0/2889939906 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f640c5970 0x7f4f64161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.795+0000 7f4f68a00640 1 -- 172.21.15.138:0/2889939906 >> v2:172.21.15.152:3300/0 conn(0x7f4f64157ea0 msgr2=0x7f4f64158280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:21.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.795+0000 7f4f68a00640 1 --2- 172.21.15.138:0/2889939906 >> v2:172.21.15.152:3300/0 conn(0x7f4f64157ea0 0x7f4f64158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.795+0000 7f4f68a00640 1 -- 172.21.15.138:0/2889939906 --> v1:172.21.15.138:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6415e5a0 con 0x7f4f6406b9b0 2024-01-21T22:25:21.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.795+0000 7f4f68a00640 1 -- 172.21.15.138:0/2889939906 <== mon.0 v1:172.21.15.138:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1288616872 0 0) 0x7f4f4c001670 con 0x7f4f6406b9b0 2024-01-21T22:25:21.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.795+0000 7f4f68a00640 1 -- 172.21.15.138:0/2889939906 >> v1:172.21.15.138:6789/0 conn(0x7f4f6406b9b0 legacy=0x7f4f6406c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:21.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.795+0000 7f4f68a00640 1 --2- 172.21.15.138:0/2889939906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f44003970 0x7f4f44003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.795+0000 7f4f68a00640 1 --2- 172.21.15.138:0/2889939906 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f640c5970 0x7f4f440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.795+0000 7f4f68a00640 1 --2- 172.21.15.138:0/2889939906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f44004e20 0x7f4f4400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.795+0000 7f4f68a00640 1 -- 172.21.15.138:0/2889939906 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6415dfe0 con 0x7f4f44003970 2024-01-21T22:25:21.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.795+0000 7f4f68a00640 1 -- 172.21.15.138:0/2889939906 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6415e2c0 con 0x7f4f44004e20 2024-01-21T22:25:21.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.795+0000 7f4f68a00640 1 -- 172.21.15.138:0/2889939906 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f440055e0 con 0x7f4f640c5970 2024-01-21T22:25:21.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.799+0000 7f4f69e00640 1 --2- 172.21.15.138:0/2889939906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f44003970 0x7f4f44003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:21.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.803+0000 7f4f69e00640 1 -- 172.21.15.138:0/2889939906 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f640c5970 msgr2=0x7f4f440047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:21.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.803+0000 7f4f69e00640 1 --2- 172.21.15.138:0/2889939906 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f640c5970 0x7f4f440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.803+0000 7f4f69e00640 1 -- 172.21.15.138:0/2889939906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f44004e20 msgr2=0x7f4f4400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:21.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.803+0000 7f4f69e00640 1 --2- 172.21.15.138:0/2889939906 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f44004e20 0x7f4f4400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.803+0000 7f4f69e00640 1 -- 172.21.15.138:0/2889939906 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f44003520 con 0x7f4f44003970 2024-01-21T22:25:21.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.805+0000 7f5f22a00640 1 -- 172.21.15.138:0/1038868746 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f5f1c013470 con 0x7f5f2c06c5d0 2024-01-21T22:25:21.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.806+0000 7f5f22a00640 1 --2- 172.21.15.138:0/1038868746 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f5f0406b3b0 0x7f5f0406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.806+0000 7f4f69e00640 1 --2- 172.21.15.138:0/2889939906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f44003970 0x7f4f44003d90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009860 tx=0x7f4f440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8a5504157e4e17 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:21.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.807+0000 7f5f2a000640 1 --2- 172.21.15.138:0/1038868746 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f5f0406b3b0 0x7f5f0406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:21.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.807+0000 7f5f2a000640 1 --2- 172.21.15.138:0/1038868746 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f5f0406b3b0 0x7f5f0406d870 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5f14036000 tx=0x7f5f14008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:21.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.808+0000 7f5f22a00640 1 -- 172.21.15.138:0/1038868746 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f1c058f00 con 0x7f5f2c06c5d0 2024-01-21T22:25:21.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.811+0000 7f4f68a00640 1 -- 172.21.15.138:0/2889939906 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f54032600 con 0x7f4f44003970 2024-01-21T22:25:21.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.811+0000 7f4f68a00640 1 -- 172.21.15.138:0/2889939906 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5402e050 con 0x7f4f44003970 2024-01-21T22:25:21.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.811+0000 7f4f68a00640 1 -- 172.21.15.138:0/2889939906 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f540329c0 con 0x7f4f44003970 2024-01-21T22:25:21.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.812+0000 7f4f6c800640 1 -- 172.21.15.138:0/2889939906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f44003970 msgr2=0x7f4f44003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:21.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.812+0000 7f4f6c800640 1 --2- 172.21.15.138:0/2889939906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f44003970 0x7f4f44003d90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009860 tx=0x7f4f440057f0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.812+0000 7f4f6a800640 1 -- 172.21.15.138:0/2889939906 reap_dead start 2024-01-21T22:25:21.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.812+0000 7f4f6c800640 1 -- 172.21.15.138:0/2889939906 shutdown_connections 2024-01-21T22:25:21.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.812+0000 7f4f6c800640 1 -- 172.21.15.138:0/2889939906 >> 172.21.15.138:0/2889939906 conn(0x7f4f640cb070 msgr2=0x7f4f640cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:21.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.813+0000 7f4f6c800640 1 -- 172.21.15.138:0/2889939906 shutdown_connections 2024-01-21T22:25:21.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.813+0000 7f4f6c800640 1 -- 172.21.15.138:0/2889939906 wait complete. 2024-01-21T22:25:21.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.815+0000 7f4f6c800640 1 Processor -- start 2024-01-21T22:25:21.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.815+0000 7f4f6c800640 1 -- start start 2024-01-21T22:25:21.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.816+0000 7f4f6c800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f44003970 0x7f4f6406a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.816+0000 7f4f6c800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f64158840 0x7f4f6415a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.816+0000 7f4f6c800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f6415a750 0x7f4f641f3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.816+0000 7f4f6c800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f640caef0 con 0x7f4f44003970 2024-01-21T22:25:21.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.816+0000 7f4f6c800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6415ac00 con 0x7f4f6415a750 2024-01-21T22:25:21.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.816+0000 7f4f6c800640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6415f210 con 0x7f4f64158840 2024-01-21T22:25:21.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.817+0000 7f4f69e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f44003970 0x7f4f6406a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:21.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.817+0000 7f4f69e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f44003970 0x7f4f6406a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53086/0 (socket says 172.21.15.138:53086) 2024-01-21T22:25:21.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.817+0000 7f4f69e00640 1 -- 172.21.15.138:0/1176955651 learned_addr learned my addr 172.21.15.138:0/1176955651 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:21.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.822+0000 7f4f69e00640 1 -- 172.21.15.138:0/1176955651 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f64158840 msgr2=0x7f4f6415a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:21.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.822+0000 7f4f69e00640 1 --2- 172.21.15.138:0/1176955651 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f64158840 0x7f4f6415a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.823+0000 7f4f69e00640 1 -- 172.21.15.138:0/1176955651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f6415a750 msgr2=0x7f4f641f3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:21.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.823+0000 7f4f69e00640 1 --2- 172.21.15.138:0/1176955651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f6415a750 0x7f4f641f3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.823+0000 7f4f69e00640 1 -- 172.21.15.138:0/1176955651 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f640ca2e0 con 0x7f4f44003970 2024-01-21T22:25:21.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.826+0000 7f4f69e00640 1 --2- 172.21.15.138:0/1176955651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f44003970 0x7f4f6406a960 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4f54035040 tx=0x7f4f540047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:21.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.831+0000 7f4f62000640 1 -- 172.21.15.138:0/1176955651 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f54003e80 con 0x7f4f44003970 2024-01-21T22:25:21.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.831+0000 7f4f62000640 1 -- 172.21.15.138:0/1176955651 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f540044e0 con 0x7f4f44003970 2024-01-21T22:25:21.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.831+0000 7f4f62000640 1 -- 172.21.15.138:0/1176955651 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5403d620 con 0x7f4f44003970 2024-01-21T22:25:21.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.831+0000 7f4f6c800640 1 -- 172.21.15.138:0/1176955651 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f640ca5c0 con 0x7f4f44003970 2024-01-21T22:25:21.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.831+0000 7f4f6c800640 1 -- 172.21.15.138:0/1176955651 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f640cab80 con 0x7f4f44003970 2024-01-21T22:25:21.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.833+0000 7f4f6c800640 1 -- 172.21.15.138:0/1176955651 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f640cad40 con 0x7f4f44003970 2024-01-21T22:25:21.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.838+0000 7fca53200640 1 Processor -- start 2024-01-21T22:25:21.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.838+0000 7f6e37a00640 1 Processor -- start 2024-01-21T22:25:21.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.838+0000 7fca53200640 1 -- start start 2024-01-21T22:25:21.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.838+0000 7f6e37a00640 1 -- start start 2024-01-21T22:25:21.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.840+0000 7fca53200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fca4c153ef0 0x7fca4c0cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.840+0000 7fca53200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fca4c0caf80 0x7fca4c0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.840+0000 7fca53200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca4c0caba0 con 0x7fca4c0c6620 2024-01-21T22:25:21.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.840+0000 7fca53200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca4c0ca240 con 0x7fca4c153ef0 2024-01-21T22:25:21.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.840+0000 7fca53200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c15aa70 con 0x7fca4c0caf80 2024-01-21T22:25:21.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.840+0000 7f6e37a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6e300ca710 0x7f6e300c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.840+0000 7f6e37a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e30154100 con 0x7f6e300cc940 2024-01-21T22:25:21.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.840+0000 7f6e37a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e300cb5c0 con 0x7f6e30153d20 2024-01-21T22:25:21.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.840+0000 7f6e37a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e300cab20 con 0x7f6e300ca710 2024-01-21T22:25:21.850 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-21T22:25:21.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.882+0000 7fcd0a000640 1 -- 172.21.15.138:0/3535120938 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fccfc057f40 con 0x7fcd0c0c43e0 2024-01-21T22:25:21.895 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-21T22:25:21.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.893+0000 7fcd0a000640 1 -- 172.21.15.138:0/3535120938 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fccfc054280 con 0x7fcd0c0c43e0 2024-01-21T22:25:21.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.898+0000 7f6e35a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6e300ca710 0x7f6e300c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:21.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.899+0000 7fca4b400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fca4c0c6620 0x7fca4c06b800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:53576/0 (socket says 172.21.15.138:53576) 2024-01-21T22:25:21.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.899+0000 7fca4b400640 1 -- 172.21.15.138:0/3716579955 learned_addr learned my addr 172.21.15.138:0/3716579955 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:21.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.899+0000 7f6e35a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6e300ca710 0x7f6e300c43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:36378/0 (socket says 172.21.15.138:36378) 2024-01-21T22:25:21.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.899+0000 7f6e35a00640 1 -- 172.21.15.138:0/3705615829 learned_addr learned my addr 172.21.15.138:0/3705615829 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:21.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.902+0000 7f6e35a00640 1 -- 172.21.15.138:0/3705615829 >> v1:172.21.15.152:6789/0 conn(0x7f6e30153d20 legacy=0x7f6e3015a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:21.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.902+0000 7f6e35a00640 1 -- 172.21.15.138:0/3705615829 >> v1:172.21.15.138:6789/0 conn(0x7f6e300cc940 legacy=0x7f6e300cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:21.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.903+0000 7f6e35a00640 1 -- 172.21.15.138:0/3705615829 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e300cb8a0 con 0x7f6e300ca710 2024-01-21T22:25:21.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.908+0000 7fca4aa00640 1 -- 172.21.15.138:0/3716579955 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392067076 0 0) 0x7fca4c0caba0 con 0x7fca4c0c6620 2024-01-21T22:25:21.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.908+0000 7fca4aa00640 1 -- 172.21.15.138:0/3716579955 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca2c003520 con 0x7fca4c0c6620 2024-01-21T22:25:21.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.908+0000 7f6e35a00640 1 --2- 172.21.15.138:0/3705615829 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e300ca710 0x7f6e300c43e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009e50 tx=0x7f6e2802bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=925351fdef0d0f0f server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:21.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.913+0000 7fca4aa00640 1 -- 172.21.15.138:0/3716579955 <== mon.1 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1318045841 0 0) 0x7fca2c003520 con 0x7fca4c0c6620 2024-01-21T22:25:21.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.913+0000 7fca4aa00640 1 -- 172.21.15.138:0/3716579955 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fca4c0caf80 msgr2=0x7fca4c0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:21.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.913+0000 7fca4aa00640 1 --2- 172.21.15.138:0/3716579955 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fca4c0caf80 0x7fca4c0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.913+0000 7fca4aa00640 1 -- 172.21.15.138:0/3716579955 >> v2:172.21.15.138:3300/0 conn(0x7fca4c153ef0 msgr2=0x7fca4c0cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:21.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.913+0000 7fca4aa00640 1 --2- 172.21.15.138:0/3716579955 >> v2:172.21.15.138:3300/0 conn(0x7fca4c153ef0 0x7fca4c0cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.913+0000 7fca4aa00640 1 -- 172.21.15.138:0/3716579955 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c0cb700 con 0x7fca4c0c6620 2024-01-21T22:25:21.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.913+0000 7fca4aa00640 1 -- 172.21.15.138:0/3716579955 <== mon.1 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1288616872 0 0) 0x7fca34001670 con 0x7fca4c0c6620 2024-01-21T22:25:21.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.914+0000 7fca4aa00640 1 -- 172.21.15.138:0/3716579955 >> v1:172.21.15.152:6789/0 conn(0x7fca4c0c6620 legacy=0x7fca4c06b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:21.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.914+0000 7fca4aa00640 1 --2- 172.21.15.138:0/3716579955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2c003970 0x7fca2c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.914+0000 7fca4aa00640 1 --2- 172.21.15.138:0/3716579955 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fca4c0caf80 0x7fca2c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.914+0000 7fca4aa00640 1 --2- 172.21.15.138:0/3716579955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca2c004d50 0x7fca2c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.914+0000 7fca4aa00640 1 -- 172.21.15.138:0/3716579955 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c0ca240 con 0x7fca2c003970 2024-01-21T22:25:21.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.914+0000 7fca4aa00640 1 -- 172.21.15.138:0/3716579955 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c15aa70 con 0x7fca2c004d50 2024-01-21T22:25:21.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.914+0000 7fca4aa00640 1 -- 172.21.15.138:0/3716579955 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c00bca0 con 0x7fca4c0caf80 2024-01-21T22:25:21.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.917+0000 7fca51200640 1 --2- 172.21.15.138:0/3716579955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca2c004d50 0x7fca2c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:21.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.919+0000 7f6e2f400640 1 -- 172.21.15.138:0/3705615829 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2802cb20 con 0x7f6e300ca710 2024-01-21T22:25:21.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.919+0000 7f6e2f400640 1 -- 172.21.15.138:0/3705615829 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2802cc80 con 0x7f6e300ca710 2024-01-21T22:25:21.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.919+0000 7f6e2f400640 1 -- 172.21.15.138:0/3705615829 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e280343f0 con 0x7f6e300ca710 2024-01-21T22:25:21.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.920+0000 7f6e37a00640 1 -- 172.21.15.138:0/3705615829 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e300ca710 msgr2=0x7f6e300c43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:21.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.920+0000 7f6e37a00640 1 --2- 172.21.15.138:0/3705615829 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e300ca710 0x7f6e300c43e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009e50 tx=0x7f6e2802bf80 comp rx=0 tx=0).stop 2024-01-21T22:25:21.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.920+0000 7f6e37a00640 1 -- 172.21.15.138:0/3705615829 shutdown_connections 2024-01-21T22:25:21.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.920+0000 7f6e37a00640 1 --2- 172.21.15.138:0/3705615829 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e300ca710 0x7f6e300c43e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.920+0000 7f6e37a00640 1 -- 172.21.15.138:0/3705615829 >> 172.21.15.138:0/3705615829 conn(0x7f6e3006d0d0 msgr2=0x7f6e3006af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:21.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.920+0000 7f6e37a00640 1 -- 172.21.15.138:0/3705615829 shutdown_connections 2024-01-21T22:25:21.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.921+0000 7f6e37a00640 1 -- 172.21.15.138:0/3705615829 wait complete. 2024-01-21T22:25:21.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.921+0000 7fca51200640 1 -- 172.21.15.138:0/3716579955 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fca4c0caf80 msgr2=0x7fca2c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:21.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.921+0000 7fca51200640 1 --2- 172.21.15.138:0/3716579955 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fca4c0caf80 0x7fca2c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.921+0000 7fca51200640 1 -- 172.21.15.138:0/3716579955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2c003970 msgr2=0x7fca2c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:21.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.921+0000 7fca51200640 1 --2- 172.21.15.138:0/3716579955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2c003970 0x7fca2c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.921+0000 7fca51200640 1 -- 172.21.15.138:0/3716579955 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c0cb700 con 0x7fca2c004d50 2024-01-21T22:25:21.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.922+0000 7f6e37a00640 1 Processor -- start 2024-01-21T22:25:21.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.922+0000 7f6e37a00640 1 -- start start 2024-01-21T22:25:21.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.923+0000 7f6e37a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e300ca710 0x7f6e301f1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.923+0000 7f6e37a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6e300cc940 0x7f6e300b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.923+0000 7f6e37a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e30153d20 0x7f6e301582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.923+0000 7f6e37a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e301575b0 con 0x7f6e300ca710 2024-01-21T22:25:21.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.923+0000 7f6e37a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3014c590 con 0x7f6e30153d20 2024-01-21T22:25:21.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.923+0000 7f6e37a00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e300b4b40 con 0x7f6e300cc940 2024-01-21T22:25:21.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.925+0000 7fca51200640 1 --2- 172.21.15.138:0/3716579955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca2c004d50 0x7fca2c0051d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fca44009e40 tx=0x7fca2c0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d042bc2a599df32 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:21.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.928+0000 7f6e35a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e30153d20 0x7f6e301582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:21.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.929+0000 7fca4aa00640 1 -- 172.21.15.138:0/3716579955 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4402cd90 con 0x7fca2c004d50 2024-01-21T22:25:21.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.929+0000 7fca4aa00640 1 -- 172.21.15.138:0/3716579955 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca44034440 con 0x7fca2c004d50 2024-01-21T22:25:21.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.929+0000 7fca4aa00640 1 -- 172.21.15.138:0/3716579955 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca440347f0 con 0x7fca2c004d50 2024-01-21T22:25:21.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.929+0000 7f6e35a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e30153d20 0x7f6e301582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:36392/0 (socket says 172.21.15.138:36392) 2024-01-21T22:25:21.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.929+0000 7f6e35a00640 1 -- 172.21.15.138:0/195209722 learned_addr learned my addr 172.21.15.138:0/195209722 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:21.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.930+0000 7fca53200640 1 -- 172.21.15.138:0/3716579955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca2c004d50 msgr2=0x7fca2c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:21.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.930+0000 7fca53200640 1 --2- 172.21.15.138:0/3716579955 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca2c004d50 0x7fca2c0051d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fca44009e40 tx=0x7fca2c0057f0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.930+0000 7fca51200640 1 -- 172.21.15.138:0/3716579955 reap_dead start 2024-01-21T22:25:21.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.930+0000 7fca53200640 1 -- 172.21.15.138:0/3716579955 shutdown_connections 2024-01-21T22:25:21.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.930+0000 7fca53200640 1 -- 172.21.15.138:0/3716579955 >> 172.21.15.138:0/3716579955 conn(0x7fca4c0cbca0 msgr2=0x7fca4c0cc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:21.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.930+0000 7fca53200640 1 -- 172.21.15.138:0/3716579955 shutdown_connections 2024-01-21T22:25:21.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.930+0000 7fca53200640 1 -- 172.21.15.138:0/3716579955 wait complete. 2024-01-21T22:25:21.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.932+0000 7fca53200640 1 Processor -- start 2024-01-21T22:25:21.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.932+0000 7fca53200640 1 -- start start 2024-01-21T22:25:21.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.933+0000 7fca53200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2c004d50 0x7fca4c148eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.933+0000 7fca53200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca4c148960 0x7fca4c0ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.933+0000 7fca53200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fca4c15bf50 0x7fca4c15c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.933+0000 7fca53200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c14e4f0 con 0x7fca2c004d50 2024-01-21T22:25:21.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.933+0000 7fca53200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c148750 con 0x7fca4c148960 2024-01-21T22:25:21.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.933+0000 7fca53200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4c1f0420 con 0x7fca4c15bf50 2024-01-21T22:25:21.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.933+0000 7f6e35a00640 1 -- 172.21.15.138:0/195209722 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6e300cc940 msgr2=0x7f6e300b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:21.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.933+0000 7f6e35a00640 1 --2- 172.21.15.138:0/195209722 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6e300cc940 0x7f6e300b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.933+0000 7f6e35a00640 1 -- 172.21.15.138:0/195209722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e300ca710 msgr2=0x7f6e301f1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:21.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.933+0000 7f6e35a00640 1 --2- 172.21.15.138:0/195209722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e300ca710 0x7f6e301f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.933+0000 7f6e35a00640 1 -- 172.21.15.138:0/195209722 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e28009b30 con 0x7f6e30153d20 2024-01-21T22:25:21.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.937+0000 7f6e35a00640 1 --2- 172.21.15.138:0/195209722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e30153d20 0x7f6e301582e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6e28036040 tx=0x7f6e28004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:21.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.937+0000 7fca4b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca4c148960 0x7fca4c0ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:21.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.937+0000 7fca4b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca4c148960 0x7fca4c0ce4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:36398/0 (socket says 172.21.15.138:36398) 2024-01-21T22:25:21.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.937+0000 7fca4b400640 1 -- 172.21.15.138:0/3925052844 learned_addr learned my addr 172.21.15.138:0/3925052844 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:21.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.941+0000 7f6e2cc00640 1 -- 172.21.15.138:0/195209722 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e28009510 con 0x7f6e30153d20 2024-01-21T22:25:21.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.941+0000 7f6e2cc00640 1 -- 172.21.15.138:0/195209722 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e28007e10 con 0x7f6e30153d20 2024-01-21T22:25:21.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.941+0000 7f6e2cc00640 1 -- 172.21.15.138:0/195209722 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2803e670 con 0x7f6e30153d20 2024-01-21T22:25:21.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.942+0000 7f6e37a00640 1 -- 172.21.15.138:0/195209722 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e300b4e20 con 0x7f6e30153d20 2024-01-21T22:25:21.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.943+0000 7f6e37a00640 1 -- 172.21.15.138:0/195209722 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e30157ae0 con 0x7f6e30153d20 2024-01-21T22:25:21.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.943+0000 7f6e37a00640 1 -- 172.21.15.138:0/195209722 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e3004ea30 con 0x7f6e30153d20 2024-01-21T22:25:21.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.944+0000 7fca4b400640 1 -- 172.21.15.138:0/3925052844 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fca4c15bf50 msgr2=0x7fca4c15c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:21.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.944+0000 7fca4b400640 1 --2- 172.21.15.138:0/3925052844 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fca4c15bf50 0x7fca4c15c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.944+0000 7fca4b400640 1 -- 172.21.15.138:0/3925052844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2c004d50 msgr2=0x7fca4c148eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:21.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.944+0000 7fca4b400640 1 --2- 172.21.15.138:0/3925052844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2c004d50 0x7fca4c148eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:21.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.944+0000 7fca4b400640 1 -- 172.21.15.138:0/3925052844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4c06b8b0 con 0x7fca4c148960 2024-01-21T22:25:21.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.983+0000 7f4f62000640 1 -- 172.21.15.138:0/1176955651 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f4f5403d780 con 0x7f4f44003970 2024-01-21T22:25:21.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.985+0000 7f4f62000640 1 --2- 172.21.15.138:0/1176955651 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f4f3c06b380 0x7f4f3c06d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:21.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.985+0000 7f4f69400640 1 --2- 172.21.15.138:0/1176955651 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f4f3c06b380 0x7f4f3c06d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:21.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:21.985+0000 7f4f69400640 1 --2- 172.21.15.138:0/1176955651 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f4f3c06b380 0x7f4f3c06d840 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c001eb0 tx=0x7f4f4c001600 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:22.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.023+0000 7f9722a00640 1 Processor -- start 2024-01-21T22:25:22.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.023+0000 7f9722a00640 1 -- start start 2024-01-21T22:25:22.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.024+0000 7f9722a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f971c06c5d0 0x7f971c0c6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:22.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.024+0000 7f9722a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f971c06c9b0 0x7f971c0c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:22.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.024+0000 7f9722a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f971c0c55a0 con 0x7f971c0c5970 2024-01-21T22:25:22.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.024+0000 7f9722a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f971c0c3d30 con 0x7f971c06c5d0 2024-01-21T22:25:22.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.024+0000 7f9722a00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f971c0c4310 con 0x7f971c06c9b0 2024-01-21T22:25:22.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.061+0000 7f5f22a00640 1 -- 172.21.15.138:0/1038868746 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f5f1c055240 con 0x7f5f2c06c5d0 2024-01-21T22:25:22.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.065+0000 7efc36400640 1 Processor -- start 2024-01-21T22:25:22.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.065+0000 7efc36400640 1 -- start start 2024-01-21T22:25:22.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.066+0000 7f971b400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f971c06c9b0 0x7f971c0c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:22.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.066+0000 7efc36400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efc3006c5d0 0x7efc300c6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:22.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.066+0000 7efc36400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7efc3006c9b0 0x7efc300c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:22.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.066+0000 7efc36400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc300c55a0 con 0x7efc300c5970 2024-01-21T22:25:22.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.066+0000 7efc36400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc300c3d30 con 0x7efc3006c9b0 2024-01-21T22:25:22.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.066+0000 7efc36400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc300c4310 con 0x7efc3006c5d0 2024-01-21T22:25:22.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.067+0000 7f971b400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f971c06c9b0 0x7f971c0c5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42576/0 (socket says 172.21.15.138:42576) 2024-01-21T22:25:22.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.067+0000 7f971b400640 1 -- 172.21.15.138:0/4102574808 learned_addr learned my addr 172.21.15.138:0/4102574808 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:22.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.068+0000 7efc2f400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efc3006c5d0 0x7efc300c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:22.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.070+0000 7f971b400640 1 -- 172.21.15.138:0/4102574808 >> v2:172.21.15.138:3300/0 conn(0x7f971c06c5d0 msgr2=0x7f971c0c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:22.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.070+0000 7f971b400640 1 --2- 172.21.15.138:0/4102574808 >> v2:172.21.15.138:3300/0 conn(0x7f971c06c5d0 0x7f971c0c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.070+0000 7f971b400640 1 -- 172.21.15.138:0/4102574808 >> v1:172.21.15.152:6789/0 conn(0x7f971c0c5970 legacy=0x7f971c06d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:22.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.070+0000 7f971b400640 1 -- 172.21.15.138:0/4102574808 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f971c0c28d0 con 0x7f971c06c9b0 2024-01-21T22:25:22.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.071+0000 7efc2f400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efc3006c5d0 0x7efc300c6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42580/0 (socket says 172.21.15.138:42580) 2024-01-21T22:25:22.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.071+0000 7efc2f400640 1 -- 172.21.15.138:0/2073663943 learned_addr learned my addr 172.21.15.138:0/2073663943 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:22.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.074+0000 7efc2f400640 1 -- 172.21.15.138:0/2073663943 >> v2:172.21.15.152:3300/0 conn(0x7efc3006c9b0 msgr2=0x7efc300c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:22.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.074+0000 7efc2f400640 1 --2- 172.21.15.138:0/2073663943 >> v2:172.21.15.152:3300/0 conn(0x7efc3006c9b0 0x7efc300c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.074+0000 7efc2f400640 1 -- 172.21.15.138:0/2073663943 >> v1:172.21.15.138:6789/0 conn(0x7efc300c5970 legacy=0x7efc3006d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:22.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.074+0000 7efc2f400640 1 -- 172.21.15.138:0/2073663943 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc300c28d0 con 0x7efc3006c5d0 2024-01-21T22:25:22.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.075+0000 7f971b400640 1 --2- 172.21.15.138:0/4102574808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f971c06c9b0 0x7f971c0c5060 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9704009870 tx=0x7f970402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cda613a96e14ceee server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:22.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.081+0000 7efc2f400640 1 --2- 172.21.15.138:0/2073663943 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efc3006c5d0 0x7efc300c6150 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efc20009870 tx=0x7efc2002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2e224472df23a53 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:22.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.082+0000 7f971aa00640 1 -- 172.21.15.138:0/4102574808 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9704030840 con 0x7f971c06c9b0 2024-01-21T22:25:22.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.082+0000 7f971aa00640 1 -- 172.21.15.138:0/4102574808 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97040309a0 con 0x7f971c06c9b0 2024-01-21T22:25:22.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.082+0000 7f971aa00640 1 -- 172.21.15.138:0/4102574808 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9704030d30 con 0x7f971c06c9b0 2024-01-21T22:25:22.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.082+0000 7f9722a00640 1 -- 172.21.15.138:0/4102574808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f971c06c9b0 msgr2=0x7f971c0c5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.082+0000 7f9722a00640 1 --2- 172.21.15.138:0/4102574808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f971c06c9b0 0x7f971c0c5060 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9704009870 tx=0x7f970402b980 comp rx=0 tx=0).stop 2024-01-21T22:25:22.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.082+0000 7f9722a00640 1 -- 172.21.15.138:0/4102574808 shutdown_connections 2024-01-21T22:25:22.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.082+0000 7f9722a00640 1 --2- 172.21.15.138:0/4102574808 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f971c06c9b0 0x7f971c0c5060 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.082+0000 7f9722a00640 1 --2- 172.21.15.138:0/4102574808 >> v2:172.21.15.138:3300/0 conn(0x7f971c06c5d0 0x7f971c0c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.083+0000 7f9722a00640 1 -- 172.21.15.138:0/4102574808 >> 172.21.15.138:0/4102574808 conn(0x7f971c0cb070 msgr2=0x7f971c0cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:22.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.083+0000 7f9722a00640 1 -- 172.21.15.138:0/4102574808 shutdown_connections 2024-01-21T22:25:22.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.083+0000 7f9722a00640 1 -- 172.21.15.138:0/4102574808 wait complete. 2024-01-21T22:25:22.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.084+0000 7f9722a00640 1 Processor -- start 2024-01-21T22:25:22.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.084+0000 7f9722a00640 1 -- start start 2024-01-21T22:25:22.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.085+0000 7f9722a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f971c06c5d0 0x7f971c1ee100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:22.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.085+0000 7f9722a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f971c06c9b0 0x7f971c153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:22.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.085+0000 7f9722a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f971c0c5970 0x7f971c14d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:22.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.085+0000 7f9722a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c153540 con 0x7f971c06c5d0 2024-01-21T22:25:22.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.085+0000 7f9722a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c1ee640 con 0x7f971c06c9b0 2024-01-21T22:25:22.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.085+0000 7f9722a00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f971c154ea0 con 0x7f971c0c5970 2024-01-21T22:25:22.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.085+0000 7efc2e000640 1 -- 172.21.15.138:0/2073663943 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc20030840 con 0x7efc3006c5d0 2024-01-21T22:25:22.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.085+0000 7efc2e000640 1 -- 172.21.15.138:0/2073663943 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc200309a0 con 0x7efc3006c5d0 2024-01-21T22:25:22.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.085+0000 7efc2e000640 1 -- 172.21.15.138:0/2073663943 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc20030d30 con 0x7efc3006c5d0 2024-01-21T22:25:22.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.086+0000 7efc36400640 1 -- 172.21.15.138:0/2073663943 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efc3006c5d0 msgr2=0x7efc300c6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.086+0000 7efc36400640 1 --2- 172.21.15.138:0/2073663943 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efc3006c5d0 0x7efc300c6150 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efc20009870 tx=0x7efc2002b980 comp rx=0 tx=0).stop 2024-01-21T22:25:22.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.086+0000 7efc36400640 1 -- 172.21.15.138:0/2073663943 shutdown_connections 2024-01-21T22:25:22.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.086+0000 7efc36400640 1 --2- 172.21.15.138:0/2073663943 >> v2:172.21.15.152:3300/0 conn(0x7efc3006c9b0 0x7efc300c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.086+0000 7efc36400640 1 --2- 172.21.15.138:0/2073663943 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efc3006c5d0 0x7efc300c6150 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.086+0000 7efc36400640 1 -- 172.21.15.138:0/2073663943 >> 172.21.15.138:0/2073663943 conn(0x7efc300cb070 msgr2=0x7efc300cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:22.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.087+0000 7efc36400640 1 -- 172.21.15.138:0/2073663943 shutdown_connections 2024-01-21T22:25:22.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.087+0000 7efc36400640 1 -- 172.21.15.138:0/2073663943 wait complete. 2024-01-21T22:25:22.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.087+0000 7f9720a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f971c0c5970 0x7f971c14d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:22.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.087+0000 7f9720a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f971c0c5970 0x7f971c14d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42588/0 (socket says 172.21.15.138:42588) 2024-01-21T22:25:22.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.087+0000 7f9720a00640 1 -- 172.21.15.138:0/2778815387 learned_addr learned my addr 172.21.15.138:0/2778815387 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:22.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.088+0000 7efc36400640 1 Processor -- start 2024-01-21T22:25:22.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.088+0000 7efc36400640 1 -- start start 2024-01-21T22:25:22.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.089+0000 7efc36400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc3006c5d0 0x7efc301ddc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:22.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.089+0000 7efc36400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efc3006c9b0 0x7efc300c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:22.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.089+0000 7efc36400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc300c5970 0x7efc3014d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:22.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.089+0000 7efc36400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc301df730 con 0x7efc3006c5d0 2024-01-21T22:25:22.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.089+0000 7efc36400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc301de4e0 con 0x7efc300c5970 2024-01-21T22:25:22.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.089+0000 7efc36400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc301ee9a0 con 0x7efc3006c9b0 2024-01-21T22:25:22.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.091+0000 7f9720a00640 1 -- 172.21.15.138:0/2778815387 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f971c06c9b0 msgr2=0x7f971c153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.091+0000 7f9720a00640 1 --2- 172.21.15.138:0/2778815387 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f971c06c9b0 0x7f971c153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.091+0000 7f9720a00640 1 -- 172.21.15.138:0/2778815387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f971c06c5d0 msgr2=0x7f971c1ee100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.091+0000 7f9720a00640 1 --2- 172.21.15.138:0/2778815387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f971c06c5d0 0x7f971c1ee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.091+0000 7f9720a00640 1 -- 172.21.15.138:0/2778815387 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97040094d0 con 0x7f971c0c5970 2024-01-21T22:25:22.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.094+0000 7efc2ea00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efc3006c9b0 0x7efc300c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:22.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.094+0000 7efc2ea00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efc3006c9b0 0x7efc300c73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42592/0 (socket says 172.21.15.138:42592) 2024-01-21T22:25:22.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.094+0000 7efc2ea00640 1 -- 172.21.15.138:0/517771377 learned_addr learned my addr 172.21.15.138:0/517771377 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:22.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.095+0000 7f9720a00640 1 --2- 172.21.15.138:0/2778815387 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f971c0c5970 0x7f971c14d4b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f971400b9e0 tx=0x7f971400bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:22.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.099+0000 7efc2ea00640 1 -- 172.21.15.138:0/517771377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc300c5970 msgr2=0x7efc3014d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.099+0000 7efc2ea00640 1 --2- 172.21.15.138:0/517771377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc300c5970 0x7efc3014d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.099+0000 7efc2ea00640 1 -- 172.21.15.138:0/517771377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc3006c5d0 msgr2=0x7efc301ddc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.099+0000 7efc2ea00640 1 --2- 172.21.15.138:0/517771377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc3006c5d0 0x7efc301ddc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.099+0000 7efc2ea00640 1 -- 172.21.15.138:0/517771377 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc200094d0 con 0x7efc3006c9b0 2024-01-21T22:25:22.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.102+0000 7f9713e00640 1 -- 172.21.15.138:0/2778815387 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971400cb20 con 0x7f971c0c5970 2024-01-21T22:25:22.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.102+0000 7f9713e00640 1 -- 172.21.15.138:0/2778815387 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971400cc80 con 0x7f971c0c5970 2024-01-21T22:25:22.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.102+0000 7f9713e00640 1 -- 172.21.15.138:0/2778815387 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9714014680 con 0x7f971c0c5970 2024-01-21T22:25:22.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.103+0000 7f9722a00640 1 -- 172.21.15.138:0/2778815387 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f971c154490 con 0x7f971c0c5970 2024-01-21T22:25:22.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.103+0000 7f9722a00640 1 -- 172.21.15.138:0/2778815387 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f971c155400 con 0x7f971c0c5970 2024-01-21T22:25:22.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.104+0000 7f9722a00640 1 -- 172.21.15.138:0/2778815387 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f971c04ea30 con 0x7f971c0c5970 2024-01-21T22:25:22.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.105+0000 7efc2ea00640 1 --2- 172.21.15.138:0/517771377 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efc3006c9b0 0x7efc300c73d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efc1800b4a0 tx=0x7efc1800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:22.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.125+0000 7f9713e00640 1 -- 172.21.15.138:0/2778815387 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f971401d3b0 con 0x7f971c0c5970 2024-01-21T22:25:22.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.126+0000 7f9713e00640 1 --2- 172.21.15.138:0/2778815387 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f96f406b380 0x7f96f406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:22.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.127+0000 7f971be00640 1 --2- 172.21.15.138:0/2778815387 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f96f406b380 0x7f96f406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:22.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.127+0000 7f971be00640 1 --2- 172.21.15.138:0/2778815387 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f96f406b380 0x7f96f406d840 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f970c009710 tx=0x7f970c006c90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:22.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.136+0000 7fcd14a00640 1 -- 172.21.15.138:0/3535120938 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcd0c15a2d0 con 0x7fcd0c0c43e0 2024-01-21T22:25:22.147 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-01-21T22:25:22.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.159+0000 7f4f62000640 1 -- 172.21.15.138:0/1176955651 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f5407a540 con 0x7f4f44003970 2024-01-21T22:25:22.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.160+0000 7f4f62000640 1 -- 172.21.15.138:0/1176955651 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f4f54076880 con 0x7f4f44003970 2024-01-21T22:25:22.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.202+0000 7f6e2cc00640 1 -- 172.21.15.138:0/195209722 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f6e2803e7d0 con 0x7f6e30153d20 2024-01-21T22:25:22.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.204+0000 7f6e2cc00640 1 --2- 172.21.15.138:0/195209722 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6e0806b3b0 0x7f6e0806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:22.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.204+0000 7f6e35000640 1 --2- 172.21.15.138:0/195209722 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6e0806b3b0 0x7f6e0806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:22.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.205+0000 7f6e35000640 1 --2- 172.21.15.138:0/195209722 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6e0806b3b0 0x7f6e0806d870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009650 tx=0x7f6e20006ba0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:22.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.207+0000 7f6e2cc00640 1 -- 172.21.15.138:0/195209722 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e2807be10 con 0x7f6e30153d20 2024-01-21T22:25:22.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.208+0000 7fca4b400640 1 --2- 172.21.15.138:0/3925052844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca4c148960 0x7fca4c0ce4a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fca34001050 tx=0x7fca3400cd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:22.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.222+0000 7fcd0a000640 1 -- 172.21.15.138:0/3535120938 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fccfc0578a0 con 0x7fcd0c0c43e0 2024-01-21T22:25:22.225 INFO:teuthology.orchestra.run.smithi138.stdout:60129542155 2024-01-21T22:25:22.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.223+0000 7f6e2cc00640 1 -- 172.21.15.138:0/195209722 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6e28077a50 con 0x7f6e30153d20 2024-01-21T22:25:22.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.226+0000 7fcd14a00640 1 -- 172.21.15.138:0/3535120938 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcce406b380 msgr2=0x7fcce406d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.226+0000 7fcd14a00640 1 --2- 172.21.15.138:0/3535120938 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcce406b380 0x7fcce406d840 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fccf4036000 tx=0x7fccf4008030 comp rx=0 tx=0).stop 2024-01-21T22:25:22.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.226+0000 7fcd14a00640 1 -- 172.21.15.138:0/3535120938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd0c0c43e0 msgr2=0x7fcd0c1f3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.226+0000 7fcd14a00640 1 --2- 172.21.15.138:0/3535120938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd0c0c43e0 0x7fcd0c1f3af0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00c3f0 tx=0x7fccfc00c8b0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.226+0000 7fcd14a00640 1 -- 172.21.15.138:0/3535120938 shutdown_connections 2024-01-21T22:25:22.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.226+0000 7fcd14a00640 1 --2- 172.21.15.138:0/3535120938 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcce406b380 0x7fcce406d840 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.226+0000 7fcd14a00640 1 --2- 172.21.15.138:0/3535120938 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcd0c153d20 0x7fcd0c1f1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.226+0000 7fcd14a00640 1 --2- 172.21.15.138:0/3535120938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd0c0cc940 0x7fcd0c1521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.226+0000 7fcd14a00640 1 --2- 172.21.15.138:0/3535120938 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcd0c0c43e0 0x7fcd0c1f3af0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.226+0000 7fcd14a00640 1 -- 172.21.15.138:0/3535120938 >> 172.21.15.138:0/3535120938 conn(0x7fcd0c06d0d0 msgr2=0x7fcd0c152db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:22.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.226+0000 7fcd14a00640 1 -- 172.21.15.138:0/3535120938 shutdown_connections 2024-01-21T22:25:22.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.226+0000 7fcd14a00640 1 -- 172.21.15.138:0/3535120938 wait complete. 2024-01-21T22:25:22.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.241+0000 7fca43e00640 1 -- 172.21.15.138:0/3925052844 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca340074e0 con 0x7fca4c148960 2024-01-21T22:25:22.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.241+0000 7fca43e00640 1 -- 172.21.15.138:0/3925052844 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca34007640 con 0x7fca4c148960 2024-01-21T22:25:22.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.241+0000 7fca43e00640 1 -- 172.21.15.138:0/3925052844 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca34004a00 con 0x7fca4c148960 2024-01-21T22:25:22.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.241+0000 7fca53200640 1 -- 172.21.15.138:0/3925052844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4c15c910 con 0x7fca4c148960 2024-01-21T22:25:22.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.241+0000 7fca53200640 1 -- 172.21.15.138:0/3925052844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4c1efaf0 con 0x7fca4c148960 2024-01-21T22:25:22.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.243+0000 7fca53200640 1 -- 172.21.15.138:0/3925052844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4c04ea30 con 0x7fca4c148960 2024-01-21T22:25:22.246 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542155 for osd.3 2024-01-21T22:25:22.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.273+0000 7f5f31600640 1 -- 172.21.15.138:0/1038868746 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5f2c04ea30 con 0x7f5f2c06c5d0 2024-01-21T22:25:22.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.356+0000 7f4f6c800640 1 -- 172.21.15.138:0/1176955651 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4f640696e0 con 0x7f4f44003970 2024-01-21T22:25:22.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.363+0000 7fb7a8400640 1 Processor -- start 2024-01-21T22:25:22.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.363+0000 7fb7a8400640 1 -- start start 2024-01-21T22:25:22.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.364+0000 7f4f62000640 1 -- 172.21.15.138:0/1176955651 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4f54037020 con 0x7f4f44003970 2024-01-21T22:25:22.366 INFO:teuthology.orchestra.run.smithi138.stdout:60129542154 2024-01-21T22:25:22.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.364+0000 7fb7a8400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb7a0153ef0 0x7fb7a00cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:22.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.364+0000 7fb7a8400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb7a00c6620 0x7fb7a00ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:22.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.365+0000 7fb7a8400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a00caba0 con 0x7fb7a00caf80 2024-01-21T22:25:22.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.365+0000 7fb7a8400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a00ca240 con 0x7fb7a0153ef0 2024-01-21T22:25:22.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.365+0000 7fb7a8400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a015aa70 con 0x7fb7a00c6620 2024-01-21T22:25:22.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.366+0000 7fb7a5a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb7a0153ef0 0x7fb7a00cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:22.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.366+0000 7fb7a5a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fb7a0153ef0 0x7fb7a00cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53168/0 (socket says 172.21.15.138:53168) 2024-01-21T22:25:22.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.366+0000 7fb7a5a00640 1 -- 172.21.15.138:0/3753076051 learned_addr learned my addr 172.21.15.138:0/3753076051 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:22.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.367+0000 7f4f6c800640 1 -- 172.21.15.138:0/1176955651 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f4f3c06b380 msgr2=0x7f4f3c06d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.367+0000 7f4f6c800640 1 --2- 172.21.15.138:0/1176955651 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f4f3c06b380 0x7f4f3c06d840 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c001eb0 tx=0x7f4f4c001600 comp rx=0 tx=0).stop 2024-01-21T22:25:22.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.367+0000 7f4f6c800640 1 -- 172.21.15.138:0/1176955651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f44003970 msgr2=0x7f4f6406a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.367+0000 7f4f6c800640 1 --2- 172.21.15.138:0/1176955651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f44003970 0x7f4f6406a960 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4f54035040 tx=0x7f4f540047d0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.367+0000 7f4f6c800640 1 -- 172.21.15.138:0/1176955651 shutdown_connections 2024-01-21T22:25:22.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.367+0000 7f4f6c800640 1 --2- 172.21.15.138:0/1176955651 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f4f3c06b380 0x7f4f3c06d840 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.367+0000 7f4f6c800640 1 --2- 172.21.15.138:0/1176955651 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4f6415a750 0x7f4f641f3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.367+0000 7f4f6c800640 1 --2- 172.21.15.138:0/1176955651 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4f64158840 0x7f4f6415a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.367+0000 7f4f6c800640 1 --2- 172.21.15.138:0/1176955651 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f44003970 0x7f4f6406a960 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.367+0000 7f4f6c800640 1 -- 172.21.15.138:0/1176955651 >> 172.21.15.138:0/1176955651 conn(0x7f4f640cb070 msgr2=0x7f4f640727e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:22.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.367+0000 7f4f6c800640 1 -- 172.21.15.138:0/1176955651 shutdown_connections 2024-01-21T22:25:22.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.367+0000 7f4f6c800640 1 -- 172.21.15.138:0/1176955651 wait complete. 2024-01-21T22:25:22.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.370+0000 7fb7a5a00640 1 -- 172.21.15.138:0/3753076051 >> v2:172.21.15.152:3300/0 conn(0x7fb7a00c6620 msgr2=0x7fb7a00ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:22.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.370+0000 7fb7a5a00640 1 --2- 172.21.15.138:0/3753076051 >> v2:172.21.15.152:3300/0 conn(0x7fb7a00c6620 0x7fb7a00ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.370+0000 7fb7a5a00640 1 -- 172.21.15.138:0/3753076051 >> v1:172.21.15.152:6789/0 conn(0x7fb7a00caf80 legacy=0x7fb7a006b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:22.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.370+0000 7fb7a5a00640 1 -- 172.21.15.138:0/3753076051 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a00cb700 con 0x7fb7a0153ef0 2024-01-21T22:25:22.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.370+0000 7f9713e00640 1 -- 172.21.15.138:0/2778815387 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f971400ce30 con 0x7f971c0c5970 2024-01-21T22:25:22.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.373+0000 7fb7a5a00640 1 --2- 172.21.15.138:0/3753076051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7a0153ef0 0x7fb7a00cc7c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00696e0 tx=0x7fb79002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ae7858e0051124 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:22.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.378+0000 7fb79fe00640 1 -- 172.21.15.138:0/3753076051 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790030860 con 0x7fb7a0153ef0 2024-01-21T22:25:22.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.378+0000 7fb79fe00640 1 -- 172.21.15.138:0/3753076051 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7900309c0 con 0x7fb7a0153ef0 2024-01-21T22:25:22.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.378+0000 7fb79fe00640 1 -- 172.21.15.138:0/3753076051 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790030d50 con 0x7fb7a0153ef0 2024-01-21T22:25:22.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.379+0000 7fb7a8400640 1 -- 172.21.15.138:0/3753076051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7a0153ef0 msgr2=0x7fb7a00cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.379+0000 7fb7a8400640 1 --2- 172.21.15.138:0/3753076051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7a0153ef0 0x7fb7a00cc7c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00696e0 tx=0x7fb79002b9a0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.379+0000 7fb7a8400640 1 -- 172.21.15.138:0/3753076051 shutdown_connections 2024-01-21T22:25:22.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.379+0000 7fb7a8400640 1 --2- 172.21.15.138:0/3753076051 >> v2:172.21.15.152:3300/0 conn(0x7fb7a00c6620 0x7fb7a00ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.379+0000 7fb7a8400640 1 --2- 172.21.15.138:0/3753076051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7a0153ef0 0x7fb7a00cc7c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.379+0000 7fb7a8400640 1 -- 172.21.15.138:0/3753076051 >> 172.21.15.138:0/3753076051 conn(0x7fb7a00cbca0 msgr2=0x7fb7a00cc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:22.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.379+0000 7fb7a8400640 1 -- 172.21.15.138:0/3753076051 shutdown_connections 2024-01-21T22:25:22.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.379+0000 7fb7a8400640 1 -- 172.21.15.138:0/3753076051 wait complete. 2024-01-21T22:25:22.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.381+0000 7fb7a8400640 1 Processor -- start 2024-01-21T22:25:22.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.381+0000 7fb7a8400640 1 -- start start 2024-01-21T22:25:22.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.381+0000 7efc27400640 1 -- 172.21.15.138:0/517771377 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18010610 con 0x7efc3006c9b0 2024-01-21T22:25:22.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.382+0000 7efc27400640 1 -- 172.21.15.138:0/517771377 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc18010770 con 0x7efc3006c9b0 2024-01-21T22:25:22.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.382+0000 7efc27400640 1 -- 172.21.15.138:0/517771377 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18010b20 con 0x7efc3006c9b0 2024-01-21T22:25:22.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.382+0000 7efc36400640 1 -- 172.21.15.138:0/517771377 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc301eec80 con 0x7efc3006c9b0 2024-01-21T22:25:22.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.382+0000 7efc36400640 1 -- 172.21.15.138:0/517771377 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc301ef240 con 0x7efc3006c9b0 2024-01-21T22:25:22.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.383+0000 7fb7a8400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7a00c6620 0x7fb7a0145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:22.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.383+0000 7fb7a8400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7a00caf80 0x7fb7a0146080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:22.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.383+0000 7fb7a8400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7a0153ef0 0x7fb7a0149c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:22.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.383+0000 7fb7a8400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a0145df0 con 0x7fb7a00caf80 2024-01-21T22:25:22.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.383+0000 7fb7a8400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a01465c0 con 0x7fb7a0153ef0 2024-01-21T22:25:22.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.383+0000 7fb7a8400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a00c3360 con 0x7fb7a00c6620 2024-01-21T22:25:22.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.383+0000 7efc36400640 1 -- 172.21.15.138:0/517771377 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc3004ea30 con 0x7efc3006c9b0 2024-01-21T22:25:22.386 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.1 2024-01-21T22:25:22.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.385+0000 7f9713e00640 1 -- 172.21.15.138:0/2778815387 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f9714054bd0 con 0x7f971c0c5970 2024-01-21T22:25:22.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.386+0000 7fb7a5000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7a00caf80 0x7fb7a0146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:22.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.387+0000 7fb7a5000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7a00caf80 0x7fb7a0146080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53172/0 (socket says 172.21.15.138:53172) 2024-01-21T22:25:22.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.387+0000 7fb7a5000640 1 -- 172.21.15.138:0/1441077686 learned_addr learned my addr 172.21.15.138:0/1441077686 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:22.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.391+0000 7fb7a5000640 1 -- 172.21.15.138:0/1441077686 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7a00c6620 msgr2=0x7fb7a0145630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.391+0000 7fb7a5000640 1 --2- 172.21.15.138:0/1441077686 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7a00c6620 0x7fb7a0145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.391+0000 7fb7a5000640 1 -- 172.21.15.138:0/1441077686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7a0153ef0 msgr2=0x7fb7a0149c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.391+0000 7fb7a5000640 1 --2- 172.21.15.138:0/1441077686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7a0153ef0 0x7fb7a0149c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.391+0000 7fb7a5000640 1 -- 172.21.15.138:0/1441077686 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7900094d0 con 0x7fb7a00caf80 2024-01-21T22:25:22.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.394+0000 7fb7a5000640 1 --2- 172.21.15.138:0/1441077686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7a00caf80 0x7fb7a0146080 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb78800b4a0 tx=0x7fb78800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:22.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.398+0000 7fb79d600640 1 -- 172.21.15.138:0/1441077686 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb788010610 con 0x7fb7a00caf80 2024-01-21T22:25:22.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.398+0000 7fb79d600640 1 -- 172.21.15.138:0/1441077686 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb788010770 con 0x7fb7a00caf80 2024-01-21T22:25:22.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.398+0000 7fb79d600640 1 -- 172.21.15.138:0/1441077686 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb788010b20 con 0x7fb7a00caf80 2024-01-21T22:25:22.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.398+0000 7fb7a8400640 1 -- 172.21.15.138:0/1441077686 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a00c36a0 con 0x7fb7a00caf80 2024-01-21T22:25:22.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.399+0000 7fb7a8400640 1 -- 172.21.15.138:0/1441077686 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a00cd6b0 con 0x7fb7a00caf80 2024-01-21T22:25:22.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.400+0000 7fb7a8400640 1 -- 172.21.15.138:0/1441077686 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a004ea30 con 0x7fb7a00caf80 2024-01-21T22:25:22.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.425+0000 7fb79d600640 1 -- 172.21.15.138:0/1441077686 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fb788010c80 con 0x7fb7a00caf80 2024-01-21T22:25:22.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.426+0000 7fb79d600640 1 --2- 172.21.15.138:0/1441077686 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fb77806b330 0x7fb77806d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:22.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.427+0000 7fb7a5a00640 1 --2- 172.21.15.138:0/1441077686 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fb77806b330 0x7fb77806d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:22.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.427+0000 7fb7a5a00640 1 --2- 172.21.15.138:0/1441077686 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fb77806b330 0x7fb77806d7f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb790036000 tx=0x7fb79003b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:22.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.505+0000 7f6e37a00640 1 -- 172.21.15.138:0/195209722 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6e3004ea30 con 0x7f6e30153d20 2024-01-21T22:25:22.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.513+0000 7fca43e00640 1 -- 172.21.15.138:0/3925052844 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fca34004b60 con 0x7fca4c148960 2024-01-21T22:25:22.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.514+0000 7fca43e00640 1 --2- 172.21.15.138:0/3925052844 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fca2406b380 0x7fca2406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:22.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.514+0000 7fca4be00640 1 --2- 172.21.15.138:0/3925052844 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fca2406b380 0x7fca2406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:22.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.515+0000 7fca4be00640 1 --2- 172.21.15.138:0/3925052844 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fca2406b380 0x7fca2406d840 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009690 tx=0x7fca3c006c90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:22.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.517+0000 7fca43e00640 1 -- 172.21.15.138:0/3925052844 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca34001d20 con 0x7fca4c148960 2024-01-21T22:25:22.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.538+0000 7fca43e00640 1 -- 172.21.15.138:0/3925052844 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fca340546b0 con 0x7fca4c148960 2024-01-21T22:25:22.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.541+0000 7f6e2cc00640 1 -- 172.21.15.138:0/195209722 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6e2807b070 con 0x7f6e30153d20 2024-01-21T22:25:22.544 INFO:teuthology.orchestra.run.smithi138.stdout:60129542155 2024-01-21T22:25:22.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.544+0000 7f6e37a00640 1 -- 172.21.15.138:0/195209722 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6e0806b3b0 msgr2=0x7f6e0806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.544+0000 7f6e37a00640 1 --2- 172.21.15.138:0/195209722 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6e0806b3b0 0x7f6e0806d870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009650 tx=0x7f6e20006ba0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.544+0000 7f6e37a00640 1 -- 172.21.15.138:0/195209722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e30153d20 msgr2=0x7f6e301582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.544+0000 7f6e37a00640 1 --2- 172.21.15.138:0/195209722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e30153d20 0x7f6e301582e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6e28036040 tx=0x7f6e28004a40 comp rx=0 tx=0).stop 2024-01-21T22:25:22.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.544+0000 7f6e37a00640 1 -- 172.21.15.138:0/195209722 shutdown_connections 2024-01-21T22:25:22.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.544+0000 7f6e37a00640 1 --2- 172.21.15.138:0/195209722 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6e0806b3b0 0x7f6e0806d870 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.544+0000 7f6e37a00640 1 --2- 172.21.15.138:0/195209722 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6e30153d20 0x7f6e301582e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.544+0000 7f6e37a00640 1 --2- 172.21.15.138:0/195209722 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6e300cc940 0x7f6e300b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.544+0000 7f6e37a00640 1 --2- 172.21.15.138:0/195209722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e300ca710 0x7f6e301f1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.544+0000 7f6e37a00640 1 -- 172.21.15.138:0/195209722 >> 172.21.15.138:0/195209722 conn(0x7f6e3006d0d0 msgr2=0x7f6e300cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:22.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.544+0000 7f6e37a00640 1 -- 172.21.15.138:0/195209722 shutdown_connections 2024-01-21T22:25:22.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.544+0000 7f6e37a00640 1 -- 172.21.15.138:0/195209722 wait complete. 2024-01-21T22:25:22.562 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542155 for osd.0 2024-01-21T22:25:22.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.578+0000 7f9722a00640 1 -- 172.21.15.138:0/2778815387 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f971c0c99d0 con 0x7f971c0c5970 2024-01-21T22:25:22.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.666+0000 7efc27400640 1 -- 172.21.15.138:0/517771377 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7efc18010c80 con 0x7efc3006c9b0 2024-01-21T22:25:22.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.668+0000 7efc27400640 1 --2- 172.21.15.138:0/517771377 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7efc0806b380 0x7efc0806d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:22.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.669+0000 7efc2f400640 1 --2- 172.21.15.138:0/517771377 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7efc0806b380 0x7efc0806d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:22.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.669+0000 7efc27400640 1 -- 172.21.15.138:0/517771377 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc18058c90 con 0x7efc3006c9b0 2024-01-21T22:25:22.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.670+0000 7efc2f400640 1 --2- 172.21.15.138:0/517771377 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7efc0806b380 0x7efc0806d840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efc20036000 tx=0x7efc2003b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:22.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.680+0000 7efc27400640 1 -- 172.21.15.138:0/517771377 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7efc180b50d0 con 0x7efc3006c9b0 2024-01-21T22:25:22.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.681+0000 7f5f22a00640 1 -- 172.21.15.138:0/1038868746 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5f1c021020 con 0x7f5f2c06c5d0 2024-01-21T22:25:22.684 INFO:teuthology.orchestra.run.smithi138.stdout:60129542154 2024-01-21T22:25:22.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.685+0000 7f5f31600640 1 -- 172.21.15.138:0/1038868746 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f5f0406b3b0 msgr2=0x7f5f0406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.685+0000 7f5f31600640 1 --2- 172.21.15.138:0/1038868746 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f5f0406b3b0 0x7f5f0406d870 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5f14036000 tx=0x7f5f14008030 comp rx=0 tx=0).stop 2024-01-21T22:25:22.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.685+0000 7f5f31600640 1 -- 172.21.15.138:0/1038868746 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f2c06c5d0 msgr2=0x7f5f2c0af8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.685+0000 7f5f31600640 1 --2- 172.21.15.138:0/1038868746 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f2c06c5d0 0x7f5f2c0af8c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c00e9b0 tx=0x7f5f1c00ee70 comp rx=0 tx=0).stop 2024-01-21T22:25:22.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.685+0000 7f5f31600640 1 -- 172.21.15.138:0/1038868746 shutdown_connections 2024-01-21T22:25:22.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.685+0000 7f5f31600640 1 --2- 172.21.15.138:0/1038868746 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f5f0406b3b0 0x7f5f0406d870 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.685+0000 7f5f31600640 1 --2- 172.21.15.138:0/1038868746 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f2c0c5970 0x7f5f2c0715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.685+0000 7f5f31600640 1 --2- 172.21.15.138:0/1038868746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f2c06c9b0 0x7f5f2c1f2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.685+0000 7f5f31600640 1 --2- 172.21.15.138:0/1038868746 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f2c06c5d0 0x7f5f2c0af8c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.685+0000 7f5f31600640 1 -- 172.21.15.138:0/1038868746 >> 172.21.15.138:0/1038868746 conn(0x7f5f2c0cb070 msgr2=0x7f5f2c0c9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:22.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.685+0000 7f5f31600640 1 -- 172.21.15.138:0/1038868746 shutdown_connections 2024-01-21T22:25:22.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.685+0000 7f5f31600640 1 -- 172.21.15.138:0/1038868746 wait complete. 2024-01-21T22:25:22.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.694+0000 7f9713e00640 1 -- 172.21.15.138:0/2778815387 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9714019020 con 0x7f971c0c5970 2024-01-21T22:25:22.696 INFO:teuthology.orchestra.run.smithi138.stdout:60129542153 2024-01-21T22:25:22.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.697+0000 7f9722a00640 1 -- 172.21.15.138:0/2778815387 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f96f406b380 msgr2=0x7f96f406d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.697+0000 7f9722a00640 1 --2- 172.21.15.138:0/2778815387 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f96f406b380 0x7f96f406d840 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f970c009710 tx=0x7f970c006c90 comp rx=0 tx=0).stop 2024-01-21T22:25:22.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.697+0000 7f9722a00640 1 -- 172.21.15.138:0/2778815387 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f971c0c5970 msgr2=0x7f971c14d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.697+0000 7f9722a00640 1 --2- 172.21.15.138:0/2778815387 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f971c0c5970 0x7f971c14d4b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f971400b9e0 tx=0x7f971400bea0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.698+0000 7f9722a00640 1 -- 172.21.15.138:0/2778815387 shutdown_connections 2024-01-21T22:25:22.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.698+0000 7f9722a00640 1 --2- 172.21.15.138:0/2778815387 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f96f406b380 0x7f96f406d840 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.698+0000 7f9722a00640 1 --2- 172.21.15.138:0/2778815387 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f971c0c5970 0x7f971c14d4b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.698+0000 7f9722a00640 1 --2- 172.21.15.138:0/2778815387 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f971c06c9b0 0x7f971c153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.698+0000 7f9722a00640 1 --2- 172.21.15.138:0/2778815387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f971c06c5d0 0x7f971c1ee100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.698+0000 7f9722a00640 1 -- 172.21.15.138:0/2778815387 >> 172.21.15.138:0/2778815387 conn(0x7f971c0cb070 msgr2=0x7f971c0c8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:22.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.698+0000 7f9722a00640 1 -- 172.21.15.138:0/2778815387 shutdown_connections 2024-01-21T22:25:22.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.698+0000 7f9722a00640 1 -- 172.21.15.138:0/2778815387 wait complete. 2024-01-21T22:25:22.705 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.4 2024-01-21T22:25:22.718 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.2 2024-01-21T22:25:22.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.729+0000 7fb79d600640 1 -- 172.21.15.138:0/1441077686 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb788058ce0 con 0x7fb7a00caf80 2024-01-21T22:25:22.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.730+0000 7fb79d600640 1 -- 172.21.15.138:0/1441077686 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fb788054920 con 0x7fb7a00caf80 2024-01-21T22:25:22.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.773+0000 7fca53200640 1 -- 172.21.15.138:0/3925052844 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fca4c06bed0 con 0x7fca4c148960 2024-01-21T22:25:22.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.809+0000 7fca43e00640 1 -- 172.21.15.138:0/3925052844 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fca34018020 con 0x7fca4c148960 2024-01-21T22:25:22.812 INFO:teuthology.orchestra.run.smithi138.stdout:60129542154 2024-01-21T22:25:22.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.812+0000 7fca53200640 1 -- 172.21.15.138:0/3925052844 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fca2406b380 msgr2=0x7fca2406d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.812+0000 7fca53200640 1 --2- 172.21.15.138:0/3925052844 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fca2406b380 0x7fca2406d840 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fca3c009690 tx=0x7fca3c006c90 comp rx=0 tx=0).stop 2024-01-21T22:25:22.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.812+0000 7fca53200640 1 -- 172.21.15.138:0/3925052844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca4c148960 msgr2=0x7fca4c0ce4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.812+0000 7fca53200640 1 --2- 172.21.15.138:0/3925052844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca4c148960 0x7fca4c0ce4a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fca34001050 tx=0x7fca3400cd20 comp rx=0 tx=0).stop 2024-01-21T22:25:22.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.813+0000 7fca53200640 1 -- 172.21.15.138:0/3925052844 shutdown_connections 2024-01-21T22:25:22.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.813+0000 7fca53200640 1 --2- 172.21.15.138:0/3925052844 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fca2406b380 0x7fca2406d840 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.813+0000 7fca53200640 1 --2- 172.21.15.138:0/3925052844 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fca4c15bf50 0x7fca4c15c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.813+0000 7fca53200640 1 --2- 172.21.15.138:0/3925052844 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fca4c148960 0x7fca4c0ce4a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.813+0000 7fca53200640 1 --2- 172.21.15.138:0/3925052844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca2c004d50 0x7fca4c148eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.813+0000 7fca53200640 1 -- 172.21.15.138:0/3925052844 >> 172.21.15.138:0/3925052844 conn(0x7fca4c0cbca0 msgr2=0x7fca4c0c8530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:22.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.813+0000 7fca53200640 1 -- 172.21.15.138:0/3925052844 shutdown_connections 2024-01-21T22:25:22.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.813+0000 7fca53200640 1 -- 172.21.15.138:0/3925052844 wait complete. 2024-01-21T22:25:22.832 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.6 2024-01-21T22:25:22.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.895+0000 7efc36400640 1 -- 172.21.15.138:0/517771377 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7efc30150fa0 con 0x7efc3006c9b0 2024-01-21T22:25:22.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.904+0000 7efc27400640 1 -- 172.21.15.138:0/517771377 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efc1801e020 con 0x7efc3006c9b0 2024-01-21T22:25:22.906 INFO:teuthology.orchestra.run.smithi138.stdout:60129542154 2024-01-21T22:25:22.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.906+0000 7efc36400640 1 -- 172.21.15.138:0/517771377 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7efc0806b380 msgr2=0x7efc0806d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.906+0000 7efc36400640 1 --2- 172.21.15.138:0/517771377 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7efc0806b380 0x7efc0806d840 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efc20036000 tx=0x7efc2003b040 comp rx=0 tx=0).stop 2024-01-21T22:25:22.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.906+0000 7efc36400640 1 -- 172.21.15.138:0/517771377 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efc3006c9b0 msgr2=0x7efc300c73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.906+0000 7efc36400640 1 --2- 172.21.15.138:0/517771377 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efc3006c9b0 0x7efc300c73d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efc1800b4a0 tx=0x7efc1800b960 comp rx=0 tx=0).stop 2024-01-21T22:25:22.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.906+0000 7efc36400640 1 -- 172.21.15.138:0/517771377 shutdown_connections 2024-01-21T22:25:22.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.906+0000 7efc36400640 1 --2- 172.21.15.138:0/517771377 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7efc0806b380 0x7efc0806d840 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.906+0000 7efc36400640 1 --2- 172.21.15.138:0/517771377 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7efc300c5970 0x7efc3014d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.906+0000 7efc36400640 1 --2- 172.21.15.138:0/517771377 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7efc3006c9b0 0x7efc300c73d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.906+0000 7efc36400640 1 --2- 172.21.15.138:0/517771377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc3006c5d0 0x7efc301ddc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.906+0000 7efc36400640 1 -- 172.21.15.138:0/517771377 >> 172.21.15.138:0/517771377 conn(0x7efc300cb070 msgr2=0x7efc300c8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:22.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.906+0000 7efc36400640 1 -- 172.21.15.138:0/517771377 shutdown_connections 2024-01-21T22:25:22.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.906+0000 7efc36400640 1 -- 172.21.15.138:0/517771377 wait complete. 2024-01-21T22:25:22.924 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.5 2024-01-21T22:25:22.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.930+0000 7fb7a8400640 1 -- 172.21.15.138:0/1441077686 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb7a00ca980 con 0x7fb7a00caf80 2024-01-21T22:25:22.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.962+0000 7fb79d600640 1 -- 172.21.15.138:0/1441077686 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb78801e020 con 0x7fb7a00caf80 2024-01-21T22:25:22.964 INFO:teuthology.orchestra.run.smithi138.stdout:60129542155 2024-01-21T22:25:22.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.964+0000 7fb7a8400640 1 -- 172.21.15.138:0/1441077686 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fb77806b330 msgr2=0x7fb77806d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.964+0000 7fb7a8400640 1 --2- 172.21.15.138:0/1441077686 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fb77806b330 0x7fb77806d7f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb790036000 tx=0x7fb79003b040 comp rx=0 tx=0).stop 2024-01-21T22:25:22.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.965+0000 7fb7a8400640 1 -- 172.21.15.138:0/1441077686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7a00caf80 msgr2=0x7fb7a0146080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:22.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.965+0000 7fb7a8400640 1 --2- 172.21.15.138:0/1441077686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7a00caf80 0x7fb7a0146080 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb78800b4a0 tx=0x7fb78800b960 comp rx=0 tx=0).stop 2024-01-21T22:25:22.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.965+0000 7fb7a8400640 1 -- 172.21.15.138:0/1441077686 shutdown_connections 2024-01-21T22:25:22.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.965+0000 7fb7a8400640 1 --2- 172.21.15.138:0/1441077686 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fb77806b330 0x7fb77806d7f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.965+0000 7fb7a8400640 1 --2- 172.21.15.138:0/1441077686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb7a0153ef0 0x7fb7a0149c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.965+0000 7fb7a8400640 1 --2- 172.21.15.138:0/1441077686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7a00caf80 0x7fb7a0146080 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.965+0000 7fb7a8400640 1 --2- 172.21.15.138:0/1441077686 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb7a00c6620 0x7fb7a0145630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:22.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.965+0000 7fb7a8400640 1 -- 172.21.15.138:0/1441077686 >> 172.21.15.138:0/1441077686 conn(0x7fb7a00cbca0 msgr2=0x7fb7a0152da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:22.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.965+0000 7fb7a8400640 1 -- 172.21.15.138:0/1441077686 shutdown_connections 2024-01-21T22:25:22.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:22.965+0000 7fb7a8400640 1 -- 172.21.15.138:0/1441077686 wait complete. 2024-01-21T22:25:22.984 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542155 for osd.7 2024-01-21T22:25:22.984 DEBUG:teuthology.parallel:result is None 2024-01-21T22:25:23.246 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-01-21T22:25:23.387 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-01-21T22:25:23.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.457+0000 7f9e1ec00640 1 Processor -- start 2024-01-21T22:25:23.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.458+0000 7f9e1ec00640 1 -- start start 2024-01-21T22:25:23.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.459+0000 7f9e1ec00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9e180c6620 0x7f9e180cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.459+0000 7f9e1ec00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9e180caf80 0x7f9e180ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.459+0000 7f9e1ec00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e180caba0 con 0x7f9e18153ef0 2024-01-21T22:25:23.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.459+0000 7f9e1ec00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e180ca240 con 0x7f9e180caf80 2024-01-21T22:25:23.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.459+0000 7f9e1ec00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1815aa70 con 0x7f9e180c6620 2024-01-21T22:25:23.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.459+0000 7f9e1cc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9e180caf80 0x7f9e180ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:23.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.460+0000 7f9e17400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9e180c6620 0x7f9e180cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:23.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.460+0000 7f9e17400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9e180c6620 0x7f9e180cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42620/0 (socket says 172.21.15.138:42620) 2024-01-21T22:25:23.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.460+0000 7f9e17400640 1 -- 172.21.15.138:0/323215308 learned_addr learned my addr 172.21.15.138:0/323215308 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:23.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.462+0000 7f9e17400640 1 -- 172.21.15.138:0/323215308 >> v2:172.21.15.138:3300/0 conn(0x7f9e180caf80 msgr2=0x7f9e180ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:23.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.462+0000 7f9e17400640 1 --2- 172.21.15.138:0/323215308 >> v2:172.21.15.138:3300/0 conn(0x7f9e180caf80 0x7f9e180ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:23.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.462+0000 7f9e17400640 1 -- 172.21.15.138:0/323215308 >> v1:172.21.15.138:6789/0 conn(0x7f9e18153ef0 legacy=0x7f9e1806b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:23.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.462+0000 7f9e17400640 1 -- 172.21.15.138:0/323215308 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e180cb700 con 0x7f9e180c6620 2024-01-21T22:25:23.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.464+0000 7f9e17400640 1 --2- 172.21.15.138:0/323215308 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9e180c6620 0x7f9e180cc7c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009870 tx=0x7f9e0002ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c23ecf864a5f770 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:23.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.468+0000 7f9e16a00640 1 -- 172.21.15.138:0/323215308 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0002f830 con 0x7f9e180c6620 2024-01-21T22:25:23.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.468+0000 7f9e16a00640 1 -- 172.21.15.138:0/323215308 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0002f990 con 0x7f9e180c6620 2024-01-21T22:25:23.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.468+0000 7f9e16a00640 1 -- 172.21.15.138:0/323215308 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0002fd20 con 0x7f9e180c6620 2024-01-21T22:25:23.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.469+0000 7f9e1ec00640 1 -- 172.21.15.138:0/323215308 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9e180c6620 msgr2=0x7f9e180cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:23.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.469+0000 7f9e1ec00640 1 --2- 172.21.15.138:0/323215308 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9e180c6620 0x7f9e180cc7c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009870 tx=0x7f9e0002ba10 comp rx=0 tx=0).stop 2024-01-21T22:25:23.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.469+0000 7f9e1ec00640 1 -- 172.21.15.138:0/323215308 shutdown_connections 2024-01-21T22:25:23.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.469+0000 7f9e1ec00640 1 --2- 172.21.15.138:0/323215308 >> v2:172.21.15.138:3300/0 conn(0x7f9e180caf80 0x7f9e180ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:23.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.469+0000 7f9e1ec00640 1 --2- 172.21.15.138:0/323215308 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9e180c6620 0x7f9e180cc7c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:23.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.469+0000 7f9e1ec00640 1 -- 172.21.15.138:0/323215308 >> 172.21.15.138:0/323215308 conn(0x7f9e180cbca0 msgr2=0x7f9e180cc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:23.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.469+0000 7f9e1ec00640 1 -- 172.21.15.138:0/323215308 shutdown_connections 2024-01-21T22:25:23.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.469+0000 7f9e1ec00640 1 -- 172.21.15.138:0/323215308 wait complete. 2024-01-21T22:25:23.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.471+0000 7f9e1ec00640 1 Processor -- start 2024-01-21T22:25:23.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.471+0000 7f9e1ec00640 1 -- start start 2024-01-21T22:25:23.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.472+0000 7f9e1ec00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e180c6620 0x7f9e18146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.472+0000 7f9e1ec00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9e180caf80 0x7f9e18147870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.472+0000 7f9e1ec00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e18153ef0 0x7f9e18147db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.472+0000 7f9e1ec00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e181f03b0 con 0x7f9e180c6620 2024-01-21T22:25:23.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.472+0000 7f9e1ec00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e181f0670 con 0x7f9e18153ef0 2024-01-21T22:25:23.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.472+0000 7f9e1ec00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e181f0950 con 0x7f9e180caf80 2024-01-21T22:25:23.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.473+0000 7f9e17e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e180c6620 0x7f9e18146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:23.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.473+0000 7f9e1cc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e18153ef0 0x7f9e18147db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:23.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.473+0000 7f9e17400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9e180caf80 0x7f9e18147870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:23.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.473+0000 7f9e17e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e180c6620 0x7f9e18146e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53190/0 (socket says 172.21.15.138:53190) 2024-01-21T22:25:23.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.473+0000 7f9e17e00640 1 -- 172.21.15.138:0/920188705 learned_addr learned my addr 172.21.15.138:0/920188705 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:23.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.477+0000 7f9e17e00640 1 -- 172.21.15.138:0/920188705 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9e180caf80 msgr2=0x7f9e18147870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:23.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.477+0000 7f9e17e00640 1 --2- 172.21.15.138:0/920188705 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9e180caf80 0x7f9e18147870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:23.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.477+0000 7f9e17e00640 1 -- 172.21.15.138:0/920188705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e18153ef0 msgr2=0x7f9e18147db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:23.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.477+0000 7f9e17e00640 1 --2- 172.21.15.138:0/920188705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e18153ef0 0x7f9e18147db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:23.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.477+0000 7f9e17e00640 1 -- 172.21.15.138:0/920188705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e000094d0 con 0x7f9e180c6620 2024-01-21T22:25:23.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.479+0000 7f9e17e00640 1 --2- 172.21.15.138:0/920188705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e180c6620 0x7f9e18146e20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800c3c0 tx=0x7f9e0800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:23.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.483+0000 7f9e0fe00640 1 -- 172.21.15.138:0/920188705 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e080052d0 con 0x7f9e180c6620 2024-01-21T22:25:23.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.483+0000 7f9e0fe00640 1 -- 172.21.15.138:0/920188705 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e08005450 con 0x7f9e180c6620 2024-01-21T22:25:23.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.483+0000 7f9e0fe00640 1 -- 172.21.15.138:0/920188705 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e08007260 con 0x7f9e180c6620 2024-01-21T22:25:23.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.483+0000 7f9e1ec00640 1 -- 172.21.15.138:0/920188705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e181f0c30 con 0x7f9e180c6620 2024-01-21T22:25:23.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.483+0000 7f9e1ec00640 1 -- 172.21.15.138:0/920188705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e1814add0 con 0x7f9e180c6620 2024-01-21T22:25:23.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.484+0000 7f9e1ec00640 1 -- 172.21.15.138:0/920188705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e181517e0 con 0x7f9e180c6620 2024-01-21T22:25:23.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.504+0000 7f9e0fe00640 1 -- 172.21.15.138:0/920188705 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f9e080073c0 con 0x7f9e180c6620 2024-01-21T22:25:23.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.505+0000 7f9e0fe00640 1 --2- 172.21.15.138:0/920188705 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f9df006b380 0x7f9df006d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.506+0000 7f9e17400640 1 --2- 172.21.15.138:0/920188705 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f9df006b380 0x7f9df006d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:23.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.507+0000 7f9e17400640 1 --2- 172.21.15.138:0/920188705 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f9df006b380 0x7f9df006d840 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9e00036000 tx=0x7f9e00008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:23.563 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-01-21T22:25:23.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.571+0000 7fbb5a600640 1 Processor -- start 2024-01-21T22:25:23.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.572+0000 7fbb5a600640 1 -- start start 2024-01-21T22:25:23.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.573+0000 7fbb5a600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fbb5406c5d0 0x7fbb540c6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.573+0000 7fbb5a600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb5406c9b0 0x7fbb540c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.573+0000 7fbb5a600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb540c55a0 con 0x7fbb540c5970 2024-01-21T22:25:23.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.573+0000 7fbb5a600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb540c3d30 con 0x7fbb5406c5d0 2024-01-21T22:25:23.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.573+0000 7fbb5a600640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540c4310 con 0x7fbb5406c9b0 2024-01-21T22:25:23.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.574+0000 7fbb4a800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb5406c9b0 0x7fbb540c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:23.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.574+0000 7fbb4a800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb5406c9b0 0x7fbb540c5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42628/0 (socket says 172.21.15.138:42628) 2024-01-21T22:25:23.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.574+0000 7fbb4a800640 1 -- 172.21.15.138:0/2838660939 learned_addr learned my addr 172.21.15.138:0/2838660939 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:23.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.576+0000 7fbb4a800640 1 -- 172.21.15.138:0/2838660939 >> v2:172.21.15.138:3300/0 conn(0x7fbb5406c5d0 msgr2=0x7fbb540c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:23.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.576+0000 7fbb4a800640 1 --2- 172.21.15.138:0/2838660939 >> v2:172.21.15.138:3300/0 conn(0x7fbb5406c5d0 0x7fbb540c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:23.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.576+0000 7fbb4a800640 1 -- 172.21.15.138:0/2838660939 >> v1:172.21.15.138:6789/0 conn(0x7fbb540c5970 legacy=0x7fbb5406d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:23.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.576+0000 7fbb4a800640 1 -- 172.21.15.138:0/2838660939 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb540c28d0 con 0x7fbb5406c9b0 2024-01-21T22:25:23.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.579+0000 7fbb4a800640 1 --2- 172.21.15.138:0/2838660939 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb5406c9b0 0x7fbb540c5060 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009870 tx=0x7fbb3c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3492cf1e526d754 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:23.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.583+0000 7fbb52a00640 1 -- 172.21.15.138:0/2838660939 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c030840 con 0x7fbb5406c9b0 2024-01-21T22:25:23.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.583+0000 7fbb52a00640 1 -- 172.21.15.138:0/2838660939 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3c0309a0 con 0x7fbb5406c9b0 2024-01-21T22:25:23.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.583+0000 7fbb52a00640 1 -- 172.21.15.138:0/2838660939 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c030d30 con 0x7fbb5406c9b0 2024-01-21T22:25:23.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.583+0000 7fbb5a600640 1 -- 172.21.15.138:0/2838660939 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb5406c9b0 msgr2=0x7fbb540c5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:23.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.583+0000 7fbb5a600640 1 --2- 172.21.15.138:0/2838660939 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb5406c9b0 0x7fbb540c5060 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009870 tx=0x7fbb3c02b980 comp rx=0 tx=0).stop 2024-01-21T22:25:23.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.583+0000 7fbb5a600640 1 -- 172.21.15.138:0/2838660939 shutdown_connections 2024-01-21T22:25:23.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.583+0000 7fbb5a600640 1 --2- 172.21.15.138:0/2838660939 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb5406c9b0 0x7fbb540c5060 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:23.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.583+0000 7fbb5a600640 1 --2- 172.21.15.138:0/2838660939 >> v2:172.21.15.138:3300/0 conn(0x7fbb5406c5d0 0x7fbb540c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:23.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.583+0000 7fbb5a600640 1 -- 172.21.15.138:0/2838660939 >> 172.21.15.138:0/2838660939 conn(0x7fbb540cb070 msgr2=0x7fbb540cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:23.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.583+0000 7fbb5a600640 1 -- 172.21.15.138:0/2838660939 shutdown_connections 2024-01-21T22:25:23.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.583+0000 7fbb5a600640 1 -- 172.21.15.138:0/2838660939 wait complete. 2024-01-21T22:25:23.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.585+0000 7fbb5a600640 1 Processor -- start 2024-01-21T22:25:23.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.585+0000 7fbb5a600640 1 -- start start 2024-01-21T22:25:23.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.586+0000 7fbb5a600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb5406c5d0 0x7fbb540ab700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.586+0000 7fbb5a600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb5406c9b0 0x7fbb541519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.586+0000 7fbb5a600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb540c5970 0x7fbb54154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.586+0000 7fbb5a600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540aad10 con 0x7fbb5406c9b0 2024-01-21T22:25:23.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.586+0000 7fbb5a600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb541f3660 con 0x7fbb540c5970 2024-01-21T22:25:23.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.586+0000 7fbb5a600640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540a9900 con 0x7fbb5406c5d0 2024-01-21T22:25:23.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.587+0000 7fbb53e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb540c5970 0x7fbb54154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:23.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.587+0000 7fbb53400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb5406c5d0 0x7fbb540ab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:23.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.587+0000 7fbb53400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb5406c5d0 0x7fbb540ab700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42640/0 (socket says 172.21.15.138:42640) 2024-01-21T22:25:23.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.587+0000 7fbb53400640 1 -- 172.21.15.138:0/2895423641 learned_addr learned my addr 172.21.15.138:0/2895423641 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:23.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.590+0000 7fbb53400640 1 -- 172.21.15.138:0/2895423641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb540c5970 msgr2=0x7fbb54154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:23.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.591+0000 7fbb53400640 1 --2- 172.21.15.138:0/2895423641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb540c5970 0x7fbb54154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:23.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.591+0000 7fbb53400640 1 -- 172.21.15.138:0/2895423641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb5406c9b0 msgr2=0x7fbb541519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:23.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.591+0000 7fbb53400640 1 --2- 172.21.15.138:0/2895423641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb5406c9b0 0x7fbb541519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:23.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.591+0000 7fbb53400640 1 -- 172.21.15.138:0/2895423641 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c0094d0 con 0x7fbb5406c5d0 2024-01-21T22:25:23.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.592+0000 7f9e0fe00640 1 -- 172.21.15.138:0/920188705 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e08057de0 con 0x7f9e180c6620 2024-01-21T22:25:23.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.593+0000 7fbb53400640 1 --2- 172.21.15.138:0/2895423641 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb5406c5d0 0x7fbb540ab700 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400b4d0 tx=0x7fbb4400b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:23.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.596+0000 7fbb4be00640 1 -- 172.21.15.138:0/2895423641 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb44010870 con 0x7fbb5406c5d0 2024-01-21T22:25:23.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.596+0000 7fbb4be00640 1 -- 172.21.15.138:0/2895423641 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb440109d0 con 0x7fbb5406c5d0 2024-01-21T22:25:23.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.596+0000 7fbb4be00640 1 -- 172.21.15.138:0/2895423641 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb44010d80 con 0x7fbb5406c5d0 2024-01-21T22:25:23.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.596+0000 7fbb5a600640 1 -- 172.21.15.138:0/2895423641 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb541f2d30 con 0x7fbb5406c5d0 2024-01-21T22:25:23.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.596+0000 7fbb5a600640 1 -- 172.21.15.138:0/2895423641 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb541f3af0 con 0x7fbb5406c5d0 2024-01-21T22:25:23.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.597+0000 7fbb5a600640 1 -- 172.21.15.138:0/2895423641 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb5404ea30 con 0x7fbb5406c5d0 2024-01-21T22:25:23.706 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-01-21T22:25:23.719 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-01-21T22:25:23.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.731+0000 7f0ed5c00640 1 Processor -- start 2024-01-21T22:25:23.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.731+0000 7f0ed5c00640 1 -- start start 2024-01-21T22:25:23.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.732+0000 7f0ed5c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0ed00cc940 0x7f0ed00ca710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.732+0000 7f0ed5c00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0ed00c43e0 0x7f0ed00cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.732+0000 7f0ed5c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed0154100 con 0x7f0ed0153d20 2024-01-21T22:25:23.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.732+0000 7f0ed5c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed00ca500 con 0x7f0ed00cc940 2024-01-21T22:25:23.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.732+0000 7f0ed5c00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed00c4810 con 0x7f0ed00c43e0 2024-01-21T22:25:23.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.733+0000 7f0ece000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0ed00cc940 0x7f0ed00ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:23.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.734+0000 7f0ece000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0ed00cc940 0x7f0ed00ca710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:36500/0 (socket says 172.21.15.138:36500) 2024-01-21T22:25:23.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.734+0000 7f0ece000640 1 -- 172.21.15.138:0/376649783 learned_addr learned my addr 172.21.15.138:0/376649783 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:23.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.736+0000 7f0ece000640 1 -- 172.21.15.138:0/376649783 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0ed00c43e0 msgr2=0x7f0ed00cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:23.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.736+0000 7f0ece000640 1 --2- 172.21.15.138:0/376649783 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0ed00c43e0 0x7f0ed00cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:23.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.736+0000 7f0ece000640 1 -- 172.21.15.138:0/376649783 >> v1:172.21.15.138:6789/0 conn(0x7f0ed0153d20 legacy=0x7f0ed015a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:23.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.736+0000 7f0ece000640 1 -- 172.21.15.138:0/376649783 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed00cb5c0 con 0x7f0ed00cc940 2024-01-21T22:25:23.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.739+0000 7f0ece000640 1 --2- 172.21.15.138:0/376649783 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ed00cc940 0x7f0ed00ca710 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0ed00b55a0 tx=0x7f0eb802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55f59a6e87068e36 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:23.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.742+0000 7f0ecd600640 1 -- 172.21.15.138:0/376649783 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8030840 con 0x7f0ed00cc940 2024-01-21T22:25:23.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.742+0000 7f0ecd600640 1 -- 172.21.15.138:0/376649783 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb80309a0 con 0x7f0ed00cc940 2024-01-21T22:25:23.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.742+0000 7f0ecd600640 1 -- 172.21.15.138:0/376649783 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8030d30 con 0x7f0ed00cc940 2024-01-21T22:25:23.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.743+0000 7f0ed5c00640 1 -- 172.21.15.138:0/376649783 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ed00cc940 msgr2=0x7f0ed00ca710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:23.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.743+0000 7f0ed5c00640 1 --2- 172.21.15.138:0/376649783 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ed00cc940 0x7f0ed00ca710 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0ed00b55a0 tx=0x7f0eb802b980 comp rx=0 tx=0).stop 2024-01-21T22:25:23.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.743+0000 7f0ed5c00640 1 -- 172.21.15.138:0/376649783 shutdown_connections 2024-01-21T22:25:23.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.743+0000 7f0ed5c00640 1 --2- 172.21.15.138:0/376649783 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0ed00c43e0 0x7f0ed00cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:23.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.743+0000 7f0ed5c00640 1 --2- 172.21.15.138:0/376649783 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ed00cc940 0x7f0ed00ca710 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:23.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.743+0000 7f0ed5c00640 1 -- 172.21.15.138:0/376649783 >> 172.21.15.138:0/376649783 conn(0x7f0ed006d0d0 msgr2=0x7f0ed006af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:23.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.743+0000 7f0ed5c00640 1 -- 172.21.15.138:0/376649783 shutdown_connections 2024-01-21T22:25:23.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.743+0000 7f0ed5c00640 1 -- 172.21.15.138:0/376649783 wait complete. 2024-01-21T22:25:23.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.745+0000 7f0ed5c00640 1 Processor -- start 2024-01-21T22:25:23.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.745+0000 7f0ed5c00640 1 -- start start 2024-01-21T22:25:23.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.746+0000 7f0ed5c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ed00c43e0 0x7f0ed01575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.746+0000 7f0ed5c00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0ed00cc940 0x7f0ed015d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.746+0000 7f0ed5c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ed0153d20 0x7f0ed0159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.746+0000 7f0ed5c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed015d380 con 0x7f0ed00c43e0 2024-01-21T22:25:23.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.746+0000 7f0ed5c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed0154920 con 0x7f0ed0153d20 2024-01-21T22:25:23.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.746+0000 7f0ed5c00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed01599a0 con 0x7f0ed00cc940 2024-01-21T22:25:23.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.747+0000 7f0ecf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ed0153d20 0x7f0ed0159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:23.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.747+0000 7f0ecf400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ed0153d20 0x7f0ed0159460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:36516/0 (socket says 172.21.15.138:36516) 2024-01-21T22:25:23.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.747+0000 7f0ecf400640 1 -- 172.21.15.138:0/2324331356 learned_addr learned my addr 172.21.15.138:0/2324331356 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:23.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.750+0000 7f0ecf400640 1 -- 172.21.15.138:0/2324331356 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0ed00cc940 msgr2=0x7f0ed015d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:23.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.750+0000 7f0ecf400640 1 --2- 172.21.15.138:0/2324331356 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0ed00cc940 0x7f0ed015d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:23.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.750+0000 7f0ecf400640 1 -- 172.21.15.138:0/2324331356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ed00c43e0 msgr2=0x7f0ed01575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:23.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.750+0000 7f0ecf400640 1 --2- 172.21.15.138:0/2324331356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ed00c43e0 0x7f0ed01575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:23.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.750+0000 7f0ecf400640 1 -- 172.21.15.138:0/2324331356 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb80094d0 con 0x7f0ed0153d20 2024-01-21T22:25:23.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.752+0000 7f0ecf400640 1 --2- 172.21.15.138:0/2324331356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ed0153d20 0x7f0ed0159460 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800b970 tx=0x7f0ec800be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:23.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.754+0000 7f0ec6a00640 1 -- 172.21.15.138:0/2324331356 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec800cb40 con 0x7f0ed0153d20 2024-01-21T22:25:23.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.754+0000 7f0ed5c00640 1 -- 172.21.15.138:0/2324331356 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed0154360 con 0x7f0ed0153d20 2024-01-21T22:25:23.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.755+0000 7f0ed5c00640 1 -- 172.21.15.138:0/2324331356 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed015cd30 con 0x7f0ed0153d20 2024-01-21T22:25:23.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.756+0000 7f0ec6a00640 1 -- 172.21.15.138:0/2324331356 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec800cca0 con 0x7f0ed0153d20 2024-01-21T22:25:23.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.756+0000 7f0ed5c00640 1 -- 172.21.15.138:0/2324331356 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed004ea30 con 0x7f0ed0153d20 2024-01-21T22:25:23.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.756+0000 7f0ec6a00640 1 -- 172.21.15.138:0/2324331356 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec801e640 con 0x7f0ed0153d20 2024-01-21T22:25:23.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.759+0000 7f9e0fe00640 1 -- 172.21.15.138:0/920188705 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f9e08054120 con 0x7f9e180c6620 2024-01-21T22:25:23.835 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-01-21T22:25:23.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.858+0000 7fbb4be00640 1 -- 172.21.15.138:0/2895423641 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fbb44023480 con 0x7fbb5406c5d0 2024-01-21T22:25:23.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.859+0000 7fbb4be00640 1 --2- 172.21.15.138:0/2895423641 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fbb2c06b3b0 0x7fbb2c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.860+0000 7fbb4a800640 1 --2- 172.21.15.138:0/2895423641 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fbb2c06b3b0 0x7fbb2c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:23.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.861+0000 7fbb4a800640 1 --2- 172.21.15.138:0/2895423641 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fbb2c06b3b0 0x7fbb2c06d870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c036000 tx=0x7fbb3c03b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:23.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.863+0000 7fbb4be00640 1 -- 172.21.15.138:0/2895423641 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb44058910 con 0x7fbb5406c5d0 2024-01-21T22:25:23.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.880+0000 7fbb4be00640 1 -- 172.21.15.138:0/2895423641 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fbb44054c50 con 0x7fbb5406c5d0 2024-01-21T22:25:23.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.890+0000 7f0e7e200640 1 Processor -- start 2024-01-21T22:25:23.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.890+0000 7f0e7e200640 1 -- start start 2024-01-21T22:25:23.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.891+0000 7f0e7e200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e780c6620 0x7f0e780cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.891+0000 7f0e7e200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e780cb700 con 0x7f0e78153ef0 2024-01-21T22:25:23.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.891+0000 7f0e7e200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e780c9ba0 con 0x7f0e780caf80 2024-01-21T22:25:23.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.891+0000 7f0e7e200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e780ca240 con 0x7f0e780c6620 2024-01-21T22:25:23.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.892+0000 7f0e76a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e780c6620 0x7f0e780cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:23.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.892+0000 7f0e77400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0e78153ef0 0x7f0e7806b800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:38024/0 (socket says 172.21.15.138:38024) 2024-01-21T22:25:23.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.892+0000 7f0e77400640 1 -- 172.21.15.138:0/3961184090 learned_addr learned my addr 172.21.15.138:0/3961184090 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:23.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.894+0000 7f0e76a00640 1 -- 172.21.15.138:0/3961184090 >> v1:172.21.15.152:6789/0 conn(0x7f0e780caf80 legacy=0x7f0e78164350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:23.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.894+0000 7f0e76a00640 1 -- 172.21.15.138:0/3961184090 >> v1:172.21.15.138:6789/0 conn(0x7f0e78153ef0 legacy=0x7f0e7806b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:23.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.894+0000 7f0e76a00640 1 -- 172.21.15.138:0/3961184090 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e780cb9a0 con 0x7f0e780c6620 2024-01-21T22:25:23.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.897+0000 7f0e76a00640 1 --2- 172.21.15.138:0/3961184090 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e780c6620 0x7f0e780cc7c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0e600097d0 tx=0x7f0e6002b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1949eb3622f8453 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:23.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.900+0000 7f0e76000640 1 -- 172.21.15.138:0/3961184090 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e60030800 con 0x7f0e780c6620 2024-01-21T22:25:23.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.900+0000 7f0e76000640 1 -- 172.21.15.138:0/3961184090 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e60030960 con 0x7f0e780c6620 2024-01-21T22:25:23.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.900+0000 7f0e76000640 1 -- 172.21.15.138:0/3961184090 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e60030cf0 con 0x7f0e780c6620 2024-01-21T22:25:23.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.901+0000 7f0e7e200640 1 -- 172.21.15.138:0/3961184090 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e780c6620 msgr2=0x7f0e780cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:23.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.901+0000 7f0e7e200640 1 --2- 172.21.15.138:0/3961184090 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e780c6620 0x7f0e780cc7c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0e600097d0 tx=0x7f0e6002b940 comp rx=0 tx=0).stop 2024-01-21T22:25:23.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.901+0000 7f0e7e200640 1 -- 172.21.15.138:0/3961184090 shutdown_connections 2024-01-21T22:25:23.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.901+0000 7f0e7e200640 1 --2- 172.21.15.138:0/3961184090 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e780c6620 0x7f0e780cc7c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:23.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.901+0000 7f0e7e200640 1 -- 172.21.15.138:0/3961184090 >> 172.21.15.138:0/3961184090 conn(0x7f0e780cbca0 msgr2=0x7f0e780cc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:23.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.901+0000 7f0e7e200640 1 -- 172.21.15.138:0/3961184090 shutdown_connections 2024-01-21T22:25:23.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.901+0000 7f0e7e200640 1 -- 172.21.15.138:0/3961184090 wait complete. 2024-01-21T22:25:23.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.903+0000 7f0e7e200640 1 Processor -- start 2024-01-21T22:25:23.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.903+0000 7f0e7e200640 1 -- start start 2024-01-21T22:25:23.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.905+0000 7f0e7e200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e780c6620 0x7f0e78152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.905+0000 7f0e7e200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e780caf80 0x7f0e7815c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.905+0000 7f0e7e200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e78153ef0 0x7f0e781e4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.905+0000 7f0e7e200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e781e5560 con 0x7f0e780caf80 2024-01-21T22:25:23.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.905+0000 7f0e7e200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e781e4c50 con 0x7f0e78153ef0 2024-01-21T22:25:23.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.905+0000 7f0e7e200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e781e3150 con 0x7f0e780c6620 2024-01-21T22:25:23.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.905+0000 7f0e77400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e780c6620 0x7f0e78152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:23.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.905+0000 7f0e77400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e780c6620 0x7f0e78152540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42654/0 (socket says 172.21.15.138:42654) 2024-01-21T22:25:23.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.906+0000 7f0e77400640 1 -- 172.21.15.138:0/814472999 learned_addr learned my addr 172.21.15.138:0/814472999 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:23.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.906+0000 7fe98e800640 1 Processor -- start 2024-01-21T22:25:23.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.906+0000 7f0e76a00640 1 --2- 172.21.15.138:0/814472999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e780caf80 0x7f0e7815c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:23.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.906+0000 7fe98e800640 1 -- start start 2024-01-21T22:25:23.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.907+0000 7fe98e800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe988153ef0 0x7fe9880cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.907+0000 7fe98e800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe9880c6620 0x7fe9880ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.907+0000 7fe98e800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9880caba0 con 0x7fe9880caf80 2024-01-21T22:25:23.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.907+0000 7fe98e800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9880ca240 con 0x7fe9880c6620 2024-01-21T22:25:23.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.907+0000 7fe98e800640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98815aa70 con 0x7fe988153ef0 2024-01-21T22:25:23.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.909+0000 7f0e77400640 1 -- 172.21.15.138:0/814472999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e78153ef0 msgr2=0x7f0e781e4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:23.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.909+0000 7f0e77400640 1 --2- 172.21.15.138:0/814472999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e78153ef0 0x7f0e781e4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:23.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.909+0000 7f0e77400640 1 -- 172.21.15.138:0/814472999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e780caf80 msgr2=0x7f0e7815c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:23.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.909+0000 7f0e77400640 1 --2- 172.21.15.138:0/814472999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e780caf80 0x7f0e7815c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:23.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.909+0000 7f0e77400640 1 -- 172.21.15.138:0/814472999 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e60009400 con 0x7f0e780c6620 2024-01-21T22:25:23.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.911+0000 7fe987400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe988153ef0 0x7fe9880cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:23.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.912+0000 7f0e77400640 1 --2- 172.21.15.138:0/814472999 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e780c6620 0x7f0e78152540 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800c3f0 tx=0x7f0e6800c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:23.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.912+0000 7fe987400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe988153ef0 0x7fe9880cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42664/0 (socket says 172.21.15.138:42664) 2024-01-21T22:25:23.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.912+0000 7fe987400640 1 -- 172.21.15.138:0/2579676630 learned_addr learned my addr 172.21.15.138:0/2579676630 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:23.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.915+0000 7fe987400640 1 -- 172.21.15.138:0/2579676630 >> v2:172.21.15.152:3300/0 conn(0x7fe9880c6620 msgr2=0x7fe9880ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:23.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.915+0000 7fe987400640 1 --2- 172.21.15.138:0/2579676630 >> v2:172.21.15.152:3300/0 conn(0x7fe9880c6620 0x7fe9880ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:23.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.915+0000 7fe987400640 1 -- 172.21.15.138:0/2579676630 >> v1:172.21.15.152:6789/0 conn(0x7fe9880caf80 legacy=0x7fe98806b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:23.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.915+0000 7fe987400640 1 -- 172.21.15.138:0/2579676630 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9880cb700 con 0x7fe988153ef0 2024-01-21T22:25:23.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.919+0000 7f0e6f400640 1 -- 172.21.15.138:0/814472999 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e68005ce0 con 0x7f0e780c6620 2024-01-21T22:25:23.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.919+0000 7f0e6f400640 1 -- 172.21.15.138:0/814472999 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e68005e40 con 0x7f0e780c6620 2024-01-21T22:25:23.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.919+0000 7f0e6f400640 1 -- 172.21.15.138:0/814472999 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e680072f0 con 0x7f0e780c6620 2024-01-21T22:25:23.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.919+0000 7f0e7e200640 1 -- 172.21.15.138:0/814472999 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e78156e20 con 0x7f0e780c6620 2024-01-21T22:25:23.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.919+0000 7f0e7e200640 1 -- 172.21.15.138:0/814472999 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e781e3c30 con 0x7f0e780c6620 2024-01-21T22:25:23.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.919+0000 7fe987400640 1 --2- 172.21.15.138:0/2579676630 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe988153ef0 0x7fe9880cc7c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe978009870 tx=0x7fe97802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a82ee27e1b76ac20 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:23.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.921+0000 7f0e7e200640 1 -- 172.21.15.138:0/814472999 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e781e3a90 con 0x7f0e780c6620 2024-01-21T22:25:23.925 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-01-21T22:25:23.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.940+0000 7f0e6f400640 1 -- 172.21.15.138:0/814472999 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f0e68007450 con 0x7f0e780c6620 2024-01-21T22:25:23.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.941+0000 7f0e6f400640 1 --2- 172.21.15.138:0/814472999 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f0e5006b380 0x7f0e5006d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:23.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.941+0000 7f0e76a00640 1 --2- 172.21.15.138:0/814472999 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f0e5006b380 0x7f0e5006d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:23.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.942+0000 7f0e76a00640 1 --2- 172.21.15.138:0/814472999 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f0e5006b380 0x7f0e5006d840 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0e60036000 tx=0x7f0e60008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:23.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:23.944+0000 7f0e6f400640 1 -- 172.21.15.138:0/814472999 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e68057dd0 con 0x7f0e780c6620 2024-01-21T22:25:24.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.015+0000 7fd968c00640 1 Processor -- start 2024-01-21T22:25:24.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.016+0000 7fd968c00640 1 -- start start 2024-01-21T22:25:24.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.017+0000 7fd968c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd96415cdc0 0x7fd9640c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:24.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.017+0000 7fd968c00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd96415d1a0 0x7fd9640c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:24.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.017+0000 7fd968c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9640c4900 con 0x7fd9640ca7b0 2024-01-21T22:25:24.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.017+0000 7fd968c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9640c52d0 con 0x7fd96415cdc0 2024-01-21T22:25:24.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.017+0000 7fd968c00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd96415dc90 con 0x7fd96415d1a0 2024-01-21T22:25:24.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.018+0000 7f9e1ec00640 1 -- 172.21.15.138:0/920188705 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9e181515c0 con 0x7f9e180c6620 2024-01-21T22:25:24.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.018+0000 7fd963400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd96415cdc0 0x7fd9640c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:24.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.018+0000 7fd963400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd96415cdc0 0x7fd9640c9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53228/0 (socket says 172.21.15.138:53228) 2024-01-21T22:25:24.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.018+0000 7fd963400640 1 -- 172.21.15.138:0/1793448706 learned_addr learned my addr 172.21.15.138:0/1793448706 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:24.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.022+0000 7fd963400640 1 -- 172.21.15.138:0/1793448706 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd96415d1a0 msgr2=0x7fd9640c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:24.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.022+0000 7fd963400640 1 --2- 172.21.15.138:0/1793448706 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd96415d1a0 0x7fd9640c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.022+0000 7fd963400640 1 -- 172.21.15.138:0/1793448706 >> v1:172.21.15.138:6789/0 conn(0x7fd9640ca7b0 legacy=0x7fd96406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.022+0000 7fd963400640 1 -- 172.21.15.138:0/1793448706 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96406b4b0 con 0x7fd96415cdc0 2024-01-21T22:25:24.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.025+0000 7f0ec6a00640 1 -- 172.21.15.138:0/2324331356 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f0ec801e7a0 con 0x7f0ed0153d20 2024-01-21T22:25:24.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.027+0000 7f0ec6a00640 1 --2- 172.21.15.138:0/2324331356 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f0ea806b330 0x7f0ea806d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:24.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.027+0000 7f0ecea00640 1 --2- 172.21.15.138:0/2324331356 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f0ea806b330 0x7f0ea806d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:24.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.027+0000 7f0ec6a00640 1 -- 172.21.15.138:0/2324331356 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ec8059030 con 0x7f0ed0153d20 2024-01-21T22:25:24.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.028+0000 7f0ecea00640 1 --2- 172.21.15.138:0/2324331356 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f0ea806b330 0x7f0ea806d7f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0009650 tx=0x7f0ec0006ba0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:24.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.029+0000 7fd963400640 1 --2- 172.21.15.138:0/1793448706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd96415cdc0 0x7fd9640c9ad0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd954009870 tx=0x7fd95402ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805a211f3902bf29 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:24.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.031+0000 7f9e0fe00640 1 -- 172.21.15.138:0/920188705 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9e08020090 con 0x7f9e180c6620 2024-01-21T22:25:24.033 INFO:teuthology.orchestra.run.smithi138.stdout:60129542156 2024-01-21T22:25:24.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.034+0000 7f9e1ec00640 1 -- 172.21.15.138:0/920188705 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f9df006b380 msgr2=0x7f9df006d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.034+0000 7f9e1ec00640 1 --2- 172.21.15.138:0/920188705 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f9df006b380 0x7f9df006d840 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9e00036000 tx=0x7f9e00008030 comp rx=0 tx=0).stop 2024-01-21T22:25:24.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.034+0000 7f9e1ec00640 1 -- 172.21.15.138:0/920188705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e180c6620 msgr2=0x7f9e18146e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.034+0000 7f9e1ec00640 1 --2- 172.21.15.138:0/920188705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e180c6620 0x7f9e18146e20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800c3c0 tx=0x7f9e0800c880 comp rx=0 tx=0).stop 2024-01-21T22:25:24.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.034+0000 7f9e1ec00640 1 -- 172.21.15.138:0/920188705 shutdown_connections 2024-01-21T22:25:24.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.034+0000 7f9e1ec00640 1 --2- 172.21.15.138:0/920188705 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f9df006b380 0x7f9df006d840 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.034+0000 7f9e1ec00640 1 --2- 172.21.15.138:0/920188705 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9e18153ef0 0x7f9e18147db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.034+0000 7f9e1ec00640 1 --2- 172.21.15.138:0/920188705 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9e180caf80 0x7f9e18147870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.034+0000 7f9e1ec00640 1 --2- 172.21.15.138:0/920188705 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e180c6620 0x7f9e18146e20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.034+0000 7f9e1ec00640 1 -- 172.21.15.138:0/920188705 >> 172.21.15.138:0/920188705 conn(0x7f9e180cbca0 msgr2=0x7f9e1814d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:24.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.034+0000 7f9e1ec00640 1 -- 172.21.15.138:0/920188705 shutdown_connections 2024-01-21T22:25:24.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.034+0000 7f9e1ec00640 1 -- 172.21.15.138:0/920188705 wait complete. 2024-01-21T22:25:24.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.039+0000 7f0ec6a00640 1 -- 172.21.15.138:0/2324331356 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0ec8055370 con 0x7f0ed0153d20 2024-01-21T22:25:24.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.040+0000 7fd962000640 1 -- 172.21.15.138:0/1793448706 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95402f830 con 0x7fd96415cdc0 2024-01-21T22:25:24.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.040+0000 7fd962000640 1 -- 172.21.15.138:0/1793448706 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95402f990 con 0x7fd96415cdc0 2024-01-21T22:25:24.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.040+0000 7fd962000640 1 -- 172.21.15.138:0/1793448706 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95402fd20 con 0x7fd96415cdc0 2024-01-21T22:25:24.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.041+0000 7fd968c00640 1 -- 172.21.15.138:0/1793448706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd96415cdc0 msgr2=0x7fd9640c9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.041+0000 7fd968c00640 1 --2- 172.21.15.138:0/1793448706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd96415cdc0 0x7fd9640c9ad0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd954009870 tx=0x7fd95402ba10 comp rx=0 tx=0).stop 2024-01-21T22:25:24.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.041+0000 7fd968c00640 1 -- 172.21.15.138:0/1793448706 shutdown_connections 2024-01-21T22:25:24.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.041+0000 7fd968c00640 1 --2- 172.21.15.138:0/1793448706 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd96415d1a0 0x7fd9640c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.041+0000 7fd968c00640 1 --2- 172.21.15.138:0/1793448706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd96415cdc0 0x7fd9640c9ad0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.041+0000 7fd968c00640 1 -- 172.21.15.138:0/1793448706 >> 172.21.15.138:0/1793448706 conn(0x7fd96406d370 msgr2=0x7fd964066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:24.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.042+0000 7fd968c00640 1 -- 172.21.15.138:0/1793448706 shutdown_connections 2024-01-21T22:25:24.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.042+0000 7fd968c00640 1 -- 172.21.15.138:0/1793448706 wait complete. 2024-01-21T22:25:24.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.044+0000 7fd968c00640 1 Processor -- start 2024-01-21T22:25:24.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.044+0000 7fd968c00640 1 -- start start 2024-01-21T22:25:24.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.046+0000 7fd968c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9640ca7b0 0x7fd9641f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:24.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.046+0000 7fd968c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd96415cdc0 0x7fd9641576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:24.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.046+0000 7fd968c00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd96415d1a0 0x7fd9640c6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:24.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.046+0000 7fd968c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9640c8a40 con 0x7fd9640ca7b0 2024-01-21T22:25:24.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.046+0000 7fd968c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9640c9000 con 0x7fd96415cdc0 2024-01-21T22:25:24.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.046+0000 7fd968c00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9641e51f0 con 0x7fd96415d1a0 2024-01-21T22:25:24.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.046+0000 7fd963400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9640ca7b0 0x7fd9641f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:24.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.047+0000 7fd963400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9640ca7b0 0x7fd9641f2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53230/0 (socket says 172.21.15.138:53230) 2024-01-21T22:25:24.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.047+0000 7fd963400640 1 -- 172.21.15.138:0/2202753175 learned_addr learned my addr 172.21.15.138:0/2202753175 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:24.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.049+0000 7fd962a00640 1 --2- 172.21.15.138:0/2202753175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd96415cdc0 0x7fd9641576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:24.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.051+0000 7fd963400640 1 -- 172.21.15.138:0/2202753175 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd96415d1a0 msgr2=0x7fd9640c6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.051+0000 7fd963400640 1 --2- 172.21.15.138:0/2202753175 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd96415d1a0 0x7fd9640c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.051+0000 7fd963400640 1 -- 172.21.15.138:0/2202753175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd96415cdc0 msgr2=0x7fd9641576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.051+0000 7fd963400640 1 --2- 172.21.15.138:0/2202753175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd96415cdc0 0x7fd9641576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.051+0000 7fd963400640 1 -- 172.21.15.138:0/2202753175 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9540094d0 con 0x7fd9640ca7b0 2024-01-21T22:25:24.230 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542156 for osd.3 2024-01-21T22:25:24.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.056+0000 7fd963400640 1 --2- 172.21.15.138:0/2202753175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9640ca7b0 0x7fd9641f2af0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd954000c00 tx=0x7fd954004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:24.231 DEBUG:teuthology.parallel:result is None 2024-01-21T22:25:24.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.109+0000 7fd8c6600640 1 Processor -- start 2024-01-21T22:25:24.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.109+0000 7fd8c6600640 1 -- start start 2024-01-21T22:25:24.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.110+0000 7fd8c6600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd8c006c5d0 0x7fd8c00c6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:24.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.110+0000 7fd8c6600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd8c006c9b0 0x7fd8c00c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:24.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.110+0000 7fd8c6600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8c00c5970 0x7fd8c006d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:24.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.110+0000 7fd8c6600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c00c3d30 con 0x7fd8c006c5d0 2024-01-21T22:25:24.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.110+0000 7fd8c6600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c00c55a0 con 0x7fd8c006c9b0 2024-01-21T22:25:24.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.110+0000 7fd8c6600640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c00c4310 con 0x7fd8c00c5970 2024-01-21T22:25:24.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.122+0000 7fd8bea00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd8c006c9b0 0x7fd8c00c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:24.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.123+0000 7fd8bea00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fd8c006c9b0 0x7fd8c00c5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:36538/0 (socket says 172.21.15.138:36538) 2024-01-21T22:25:24.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.123+0000 7fd8bea00640 1 -- 172.21.15.138:0/3086532457 learned_addr learned my addr 172.21.15.138:0/3086532457 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:24.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.126+0000 7fd8bea00640 1 -- 172.21.15.138:0/3086532457 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8c00c5970 msgr2=0x7fd8c006d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:24.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.126+0000 7fd8bea00640 1 --2- 172.21.15.138:0/3086532457 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8c00c5970 0x7fd8c006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.126+0000 7fd8bea00640 1 -- 172.21.15.138:0/3086532457 >> v2:172.21.15.138:3300/0 conn(0x7fd8c006c5d0 msgr2=0x7fd8c00c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:24.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.126+0000 7fd8bea00640 1 --2- 172.21.15.138:0/3086532457 >> v2:172.21.15.138:3300/0 conn(0x7fd8c006c5d0 0x7fd8c00c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.126+0000 7fd8bea00640 1 -- 172.21.15.138:0/3086532457 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c00c28d0 con 0x7fd8c006c9b0 2024-01-21T22:25:24.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.129+0000 7fd8bea00640 1 --2- 172.21.15.138:0/3086532457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8c006c9b0 0x7fd8c00c5060 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd8c006dd00 tx=0x7fd8a802b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b986ed6cac823401 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:24.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.131+0000 7fd8be000640 1 -- 172.21.15.138:0/3086532457 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8030860 con 0x7fd8c006c9b0 2024-01-21T22:25:24.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.134+0000 7fd8be000640 1 -- 172.21.15.138:0/3086532457 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a80309c0 con 0x7fd8c006c9b0 2024-01-21T22:25:24.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.134+0000 7fd8be000640 1 -- 172.21.15.138:0/3086532457 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8030d50 con 0x7fd8c006c9b0 2024-01-21T22:25:24.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.135+0000 7fd8c6600640 1 -- 172.21.15.138:0/3086532457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8c006c9b0 msgr2=0x7fd8c00c5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.135+0000 7fd8c6600640 1 --2- 172.21.15.138:0/3086532457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8c006c9b0 0x7fd8c00c5060 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd8c006dd00 tx=0x7fd8a802b9a0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.135+0000 7fd8c6600640 1 -- 172.21.15.138:0/3086532457 shutdown_connections 2024-01-21T22:25:24.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.135+0000 7fd8c6600640 1 --2- 172.21.15.138:0/3086532457 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8c00c5970 0x7fd8c006d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.135+0000 7fd8c6600640 1 --2- 172.21.15.138:0/3086532457 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8c006c9b0 0x7fd8c00c5060 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.135+0000 7fd8c6600640 1 --2- 172.21.15.138:0/3086532457 >> v2:172.21.15.138:3300/0 conn(0x7fd8c006c5d0 0x7fd8c00c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.135+0000 7fd8c6600640 1 -- 172.21.15.138:0/3086532457 >> 172.21.15.138:0/3086532457 conn(0x7fd8c00cb070 msgr2=0x7fd8c00cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:24.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.135+0000 7fd8c6600640 1 -- 172.21.15.138:0/3086532457 shutdown_connections 2024-01-21T22:25:24.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.135+0000 7fd8c6600640 1 -- 172.21.15.138:0/3086532457 wait complete. 2024-01-21T22:25:24.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.137+0000 7fd8c6600640 1 Processor -- start 2024-01-21T22:25:24.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.137+0000 7fd8c6600640 1 -- start start 2024-01-21T22:25:24.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.138+0000 7fd8c6600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8c006c5d0 0x7fd8c00c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:24.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.138+0000 7fd8c6600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8c006c9b0 0x7fd8c00caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:24.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.138+0000 7fd8c6600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c00c5970 0x7fd8c0071350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:24.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.138+0000 7fd8c6600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c0152920 con 0x7fd8c00c5970 2024-01-21T22:25:24.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.138+0000 7fd8c6600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c01ef880 con 0x7fd8c006c9b0 2024-01-21T22:25:24.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.138+0000 7fd8c6600640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c00ca5f0 con 0x7fd8c006c5d0 2024-01-21T22:25:24.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.138+0000 7fd8bea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8c006c9b0 0x7fd8c00caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:24.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.139+0000 7fd8bea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8c006c9b0 0x7fd8c00caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:36548/0 (socket says 172.21.15.138:36548) 2024-01-21T22:25:24.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.139+0000 7fd8bea00640 1 -- 172.21.15.138:0/3416329108 learned_addr learned my addr 172.21.15.138:0/3416329108 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:24.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.142+0000 7fd8bea00640 1 -- 172.21.15.138:0/3416329108 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8c006c5d0 msgr2=0x7fd8c00c5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.142+0000 7fd8bea00640 1 --2- 172.21.15.138:0/3416329108 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8c006c5d0 0x7fd8c00c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.142+0000 7fd8bea00640 1 -- 172.21.15.138:0/3416329108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c00c5970 msgr2=0x7fd8c0071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.142+0000 7fd8bea00640 1 --2- 172.21.15.138:0/3416329108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c00c5970 0x7fd8c0071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.142+0000 7fd8bea00640 1 -- 172.21.15.138:0/3416329108 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a80094d0 con 0x7fd8c006c9b0 2024-01-21T22:25:24.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.144+0000 7fd8bea00640 1 --2- 172.21.15.138:0/3416329108 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8c006c9b0 0x7fd8c00caa80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8000c00 tx=0x7fd8a8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:24.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.148+0000 7fd95b400640 1 -- 172.21.15.138:0/2202753175 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95402f7e0 con 0x7fd9640ca7b0 2024-01-21T22:25:24.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.148+0000 7fd968c00640 1 -- 172.21.15.138:0/2202753175 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9641e5450 con 0x7fd9640ca7b0 2024-01-21T22:25:24.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.148+0000 7fd968c00640 1 -- 172.21.15.138:0/2202753175 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9641f3680 con 0x7fd9640ca7b0 2024-01-21T22:25:24.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.149+0000 7fd8b7400640 1 -- 172.21.15.138:0/3416329108 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8039d40 con 0x7fd8c006c9b0 2024-01-21T22:25:24.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.149+0000 7fd8b7400640 1 -- 172.21.15.138:0/3416329108 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a8043400 con 0x7fd8c006c9b0 2024-01-21T22:25:24.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.149+0000 7fd8b7400640 1 -- 172.21.15.138:0/3416329108 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8039ea0 con 0x7fd8c006c9b0 2024-01-21T22:25:24.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.149+0000 7fd8c6600640 1 -- 172.21.15.138:0/3416329108 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c00c9ce0 con 0x7fd8c006c9b0 2024-01-21T22:25:24.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.149+0000 7fd968c00640 1 -- 172.21.15.138:0/2202753175 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd964074ee0 con 0x7fd9640ca7b0 2024-01-21T22:25:24.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.149+0000 7fd8c6600640 1 -- 172.21.15.138:0/3416329108 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c00c9fa0 con 0x7fd8c006c9b0 2024-01-21T22:25:24.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.151+0000 7fd8c6600640 1 -- 172.21.15.138:0/3416329108 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c0069550 con 0x7fd8c006c9b0 2024-01-21T22:25:24.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.154+0000 7fd95b400640 1 -- 172.21.15.138:0/2202753175 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95404b460 con 0x7fd9640ca7b0 2024-01-21T22:25:24.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.154+0000 7fd95b400640 1 -- 172.21.15.138:0/2202753175 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95404b7d0 con 0x7fd9640ca7b0 2024-01-21T22:25:24.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.171+0000 7fd8b7400640 1 -- 172.21.15.138:0/3416329108 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fd8a80437b0 con 0x7fd8c006c9b0 2024-01-21T22:25:24.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.171+0000 7fd8b7400640 1 --2- 172.21.15.138:0/3416329108 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd89806b3b0 0x7fd89806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:24.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.172+0000 7fd8bf400640 1 --2- 172.21.15.138:0/3416329108 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd89806b3b0 0x7fd89806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:24.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.173+0000 7fd8bf400640 1 --2- 172.21.15.138:0/3416329108 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd89806b3b0 0x7fd89806d870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0009710 tx=0x7fd8b0006c90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:24.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.173+0000 7fd8b7400640 1 -- 172.21.15.138:0/3416329108 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8a803e070 con 0x7fd8c006c9b0 2024-01-21T22:25:24.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.198+0000 7fbb5a600640 1 -- 172.21.15.138:0/2895423641 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbb54150c20 con 0x7fbb5406c5d0 2024-01-21T22:25:24.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.199+0000 7f0e6f400640 1 -- 172.21.15.138:0/814472999 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0e68054110 con 0x7f0e780c6620 2024-01-21T22:25:24.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.201+0000 7fe986000640 1 -- 172.21.15.138:0/2579676630 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978030840 con 0x7fe988153ef0 2024-01-21T22:25:24.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.201+0000 7fe986000640 1 -- 172.21.15.138:0/2579676630 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9780309a0 con 0x7fe988153ef0 2024-01-21T22:25:24.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.201+0000 7fe986000640 1 -- 172.21.15.138:0/2579676630 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978030d30 con 0x7fe988153ef0 2024-01-21T22:25:24.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.202+0000 7fe98e800640 1 -- 172.21.15.138:0/2579676630 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe988153ef0 msgr2=0x7fe9880cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.202+0000 7fe98e800640 1 --2- 172.21.15.138:0/2579676630 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe988153ef0 0x7fe9880cc7c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe978009870 tx=0x7fe97802b980 comp rx=0 tx=0).stop 2024-01-21T22:25:24.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.202+0000 7fe98e800640 1 -- 172.21.15.138:0/2579676630 shutdown_connections 2024-01-21T22:25:24.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.202+0000 7fe98e800640 1 --2- 172.21.15.138:0/2579676630 >> v2:172.21.15.152:3300/0 conn(0x7fe9880c6620 0x7fe9880ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.202+0000 7fe98e800640 1 --2- 172.21.15.138:0/2579676630 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe988153ef0 0x7fe9880cc7c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.202+0000 7fe98e800640 1 -- 172.21.15.138:0/2579676630 >> 172.21.15.138:0/2579676630 conn(0x7fe9880cbca0 msgr2=0x7fe9880cc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:24.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.202+0000 7fe98e800640 1 -- 172.21.15.138:0/2579676630 shutdown_connections 2024-01-21T22:25:24.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.202+0000 7fe98e800640 1 -- 172.21.15.138:0/2579676630 wait complete. 2024-01-21T22:25:24.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.205+0000 7fe98e800640 1 Processor -- start 2024-01-21T22:25:24.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.205+0000 7fe98e800640 1 -- start start 2024-01-21T22:25:24.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.206+0000 7fe98e800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9880c6620 0x7fe9880c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:24.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.207+0000 7fe98e800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9880caf80 0x7fe9880c3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:24.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.207+0000 7fe98e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe988153ef0 0x7fe988146de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:24.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.207+0000 7fe98e800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe988145a90 con 0x7fe988153ef0 2024-01-21T22:25:24.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.207+0000 7fe98e800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9880c5d00 con 0x7fe9880caf80 2024-01-21T22:25:24.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.207+0000 7fe98e800640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9881537c0 con 0x7fe9880c6620 2024-01-21T22:25:24.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.217+0000 7fe987400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9880c6620 0x7fe9880c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:24.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.218+0000 7fe987400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9880c6620 0x7fe9880c2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42712/0 (socket says 172.21.15.138:42712) 2024-01-21T22:25:24.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.218+0000 7fe987400640 1 -- 172.21.15.138:0/3991074686 learned_addr learned my addr 172.21.15.138:0/3991074686 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:24.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.221+0000 7fe987400640 1 -- 172.21.15.138:0/3991074686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9880caf80 msgr2=0x7fe9880c3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.221+0000 7fe987400640 1 --2- 172.21.15.138:0/3991074686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9880caf80 0x7fe9880c3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.221+0000 7fe987400640 1 -- 172.21.15.138:0/3991074686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe988153ef0 msgr2=0x7fe988146de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.221+0000 7fe987400640 1 --2- 172.21.15.138:0/3991074686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe988153ef0 0x7fe988146de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.221+0000 7fe987400640 1 -- 172.21.15.138:0/3991074686 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9780094d0 con 0x7fe9880c6620 2024-01-21T22:25:24.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.284+0000 7fbb4be00640 1 -- 172.21.15.138:0/2895423641 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbb4401e070 con 0x7fbb5406c5d0 2024-01-21T22:25:24.287 INFO:teuthology.orchestra.run.smithi138.stdout:60129542155 2024-01-21T22:25:24.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.285+0000 7fe987400640 1 --2- 172.21.15.138:0/3991074686 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9880c6620 0x7fe9880c2f20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe978009990 tx=0x7fe978004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:24.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.287+0000 7fbb5a600640 1 -- 172.21.15.138:0/2895423641 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fbb2c06b3b0 msgr2=0x7fbb2c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.287+0000 7fbb5a600640 1 --2- 172.21.15.138:0/2895423641 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fbb2c06b3b0 0x7fbb2c06d870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c036000 tx=0x7fbb3c03b040 comp rx=0 tx=0).stop 2024-01-21T22:25:24.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.287+0000 7fbb5a600640 1 -- 172.21.15.138:0/2895423641 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb5406c5d0 msgr2=0x7fbb540ab700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.287+0000 7fbb5a600640 1 --2- 172.21.15.138:0/2895423641 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb5406c5d0 0x7fbb540ab700 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400b4d0 tx=0x7fbb4400b990 comp rx=0 tx=0).stop 2024-01-21T22:25:24.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.287+0000 7fbb5a600640 1 -- 172.21.15.138:0/2895423641 shutdown_connections 2024-01-21T22:25:24.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.288+0000 7fbb5a600640 1 --2- 172.21.15.138:0/2895423641 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fbb2c06b3b0 0x7fbb2c06d870 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.288+0000 7fbb5a600640 1 --2- 172.21.15.138:0/2895423641 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fbb540c5970 0x7fbb54154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.288+0000 7fbb5a600640 1 --2- 172.21.15.138:0/2895423641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb5406c9b0 0x7fbb541519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.288+0000 7fbb5a600640 1 --2- 172.21.15.138:0/2895423641 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fbb5406c5d0 0x7fbb540ab700 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.288+0000 7fbb5a600640 1 -- 172.21.15.138:0/2895423641 >> 172.21.15.138:0/2895423641 conn(0x7fbb540cb070 msgr2=0x7fbb540c8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:24.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.288+0000 7fbb5a600640 1 -- 172.21.15.138:0/2895423641 shutdown_connections 2024-01-21T22:25:24.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.288+0000 7fbb5a600640 1 -- 172.21.15.138:0/2895423641 wait complete. 2024-01-21T22:25:24.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.290+0000 7fe97f400640 1 -- 172.21.15.138:0/3991074686 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978039c20 con 0x7fe9880c6620 2024-01-21T22:25:24.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.290+0000 7fe97f400640 1 -- 172.21.15.138:0/3991074686 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe978039d80 con 0x7fe9880c6620 2024-01-21T22:25:24.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.290+0000 7fe97f400640 1 -- 172.21.15.138:0/3991074686 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978043650 con 0x7fe9880c6620 2024-01-21T22:25:24.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.290+0000 7fe98e800640 1 -- 172.21.15.138:0/3991074686 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe988153520 con 0x7fe9880c6620 2024-01-21T22:25:24.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.290+0000 7fe98e800640 1 -- 172.21.15.138:0/3991074686 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9880c5640 con 0x7fe9880c6620 2024-01-21T22:25:24.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.292+0000 7fe98e800640 1 -- 172.21.15.138:0/3991074686 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe98804ea30 con 0x7fe9880c6620 2024-01-21T22:25:24.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.299+0000 7f0ed5c00640 1 -- 172.21.15.138:0/2324331356 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0ed00c62b0 con 0x7f0ed0153d20 2024-01-21T22:25:24.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542155 for osd.1 2024-01-21T22:25:24.313 DEBUG:teuthology.parallel:result is None 2024-01-21T22:25:24.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.312+0000 7fe97f400640 1 -- 172.21.15.138:0/3991074686 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fe97804c3b0 con 0x7fe9880c6620 2024-01-21T22:25:24.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.313+0000 7fe97f400640 1 --2- 172.21.15.138:0/3991074686 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe96006b380 0x7fe96006d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:24.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.313+0000 7fe986a00640 1 --2- 172.21.15.138:0/3991074686 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe96006b380 0x7fe96006d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:24.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.314+0000 7fe986a00640 1 --2- 172.21.15.138:0/3991074686 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe96006b380 0x7fe96006d840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe98806b2e0 tx=0x7fe970006c90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:24.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.401+0000 7f0e7e200640 1 -- 172.21.15.138:0/814472999 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f0e781e3290 con 0x7f0e780c6620 2024-01-21T22:25:24.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.413+0000 7f0ec6a00640 1 -- 172.21.15.138:0/2324331356 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0ec8023090 con 0x7f0ed0153d20 2024-01-21T22:25:24.415 INFO:teuthology.orchestra.run.smithi138.stdout:60129542156 2024-01-21T22:25:24.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.413+0000 7fd95b400640 1 -- 172.21.15.138:0/2202753175 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fd95404b930 con 0x7fd9640ca7b0 2024-01-21T22:25:24.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.414+0000 7fd95b400640 1 --2- 172.21.15.138:0/2202753175 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd93c06b360 0x7fd93c06d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:24.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.414+0000 7fd962a00640 1 --2- 172.21.15.138:0/2202753175 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd93c06b360 0x7fd93c06d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:24.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.415+0000 7fd962a00640 1 --2- 172.21.15.138:0/2202753175 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd93c06b360 0x7fd93c06d820 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd9641573a0 tx=0x7fd94c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:24.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.416+0000 7f0ed5c00640 1 -- 172.21.15.138:0/2324331356 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f0ea806b330 msgr2=0x7f0ea806d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.416+0000 7f0ed5c00640 1 --2- 172.21.15.138:0/2324331356 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f0ea806b330 0x7f0ea806d7f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0009650 tx=0x7f0ec0006ba0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.416+0000 7f0ed5c00640 1 -- 172.21.15.138:0/2324331356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ed0153d20 msgr2=0x7f0ed0159460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.416+0000 7f0ed5c00640 1 --2- 172.21.15.138:0/2324331356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ed0153d20 0x7f0ed0159460 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800b970 tx=0x7f0ec800be30 comp rx=0 tx=0).stop 2024-01-21T22:25:24.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.416+0000 7f0ed5c00640 1 -- 172.21.15.138:0/2324331356 shutdown_connections 2024-01-21T22:25:24.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.416+0000 7f0ed5c00640 1 --2- 172.21.15.138:0/2324331356 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f0ea806b330 0x7f0ea806d7f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.416+0000 7f0ed5c00640 1 --2- 172.21.15.138:0/2324331356 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0ed0153d20 0x7f0ed0159460 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.416+0000 7f0ed5c00640 1 --2- 172.21.15.138:0/2324331356 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0ed00cc940 0x7f0ed015d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.416+0000 7f0ed5c00640 1 --2- 172.21.15.138:0/2324331356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ed00c43e0 0x7f0ed01575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.416+0000 7f0ed5c00640 1 -- 172.21.15.138:0/2324331356 >> 172.21.15.138:0/2324331356 conn(0x7f0ed006d0d0 msgr2=0x7f0ed0159fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:24.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.416+0000 7f0ed5c00640 1 -- 172.21.15.138:0/2324331356 shutdown_connections 2024-01-21T22:25:24.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.416+0000 7f0ed5c00640 1 -- 172.21.15.138:0/2324331356 wait complete. 2024-01-21T22:25:24.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.429+0000 7fd95b400640 1 -- 172.21.15.138:0/2202753175 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd95403c070 con 0x7fd9640ca7b0 2024-01-21T22:25:24.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.429+0000 7fd95b400640 1 -- 172.21.15.138:0/2202753175 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd9540783a0 con 0x7fd9640ca7b0 2024-01-21T22:25:24.434 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542156 for osd.0 2024-01-21T22:25:24.434 DEBUG:teuthology.parallel:result is None 2024-01-21T22:25:24.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.433+0000 7fd8b7400640 1 -- 172.21.15.138:0/3416329108 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd8a80785f0 con 0x7fd8c006c9b0 2024-01-21T22:25:24.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.553+0000 7fe97f400640 1 -- 172.21.15.138:0/3991074686 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe978044e60 con 0x7fe9880c6620 2024-01-21T22:25:24.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.565+0000 7fe97f400640 1 -- 172.21.15.138:0/3991074686 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe978077250 con 0x7fe9880c6620 2024-01-21T22:25:24.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.571+0000 7f0e6f400640 1 -- 172.21.15.138:0/814472999 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0e68017090 con 0x7f0e780c6620 2024-01-21T22:25:24.573 INFO:teuthology.orchestra.run.smithi138.stdout:60129542155 2024-01-21T22:25:24.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.575+0000 7f0e7e200640 1 -- 172.21.15.138:0/814472999 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f0e5006b380 msgr2=0x7f0e5006d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.575+0000 7f0e7e200640 1 --2- 172.21.15.138:0/814472999 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f0e5006b380 0x7f0e5006d840 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0e60036000 tx=0x7f0e60008030 comp rx=0 tx=0).stop 2024-01-21T22:25:24.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.575+0000 7f0e7e200640 1 -- 172.21.15.138:0/814472999 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e780c6620 msgr2=0x7f0e78152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.575+0000 7f0e7e200640 1 --2- 172.21.15.138:0/814472999 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e780c6620 0x7f0e78152540 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800c3f0 tx=0x7f0e6800c8b0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.575+0000 7f0e7e200640 1 -- 172.21.15.138:0/814472999 shutdown_connections 2024-01-21T22:25:24.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.575+0000 7f0e7e200640 1 --2- 172.21.15.138:0/814472999 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f0e5006b380 0x7f0e5006d840 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.575+0000 7f0e7e200640 1 --2- 172.21.15.138:0/814472999 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0e78153ef0 0x7f0e781e4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.575+0000 7f0e7e200640 1 --2- 172.21.15.138:0/814472999 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e780caf80 0x7f0e7815c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.575+0000 7f0e7e200640 1 --2- 172.21.15.138:0/814472999 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f0e780c6620 0x7f0e78152540 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.575+0000 7f0e7e200640 1 -- 172.21.15.138:0/814472999 >> 172.21.15.138:0/814472999 conn(0x7f0e780cbca0 msgr2=0x7f0e78152b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:24.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.575+0000 7f0e7e200640 1 -- 172.21.15.138:0/814472999 shutdown_connections 2024-01-21T22:25:24.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.575+0000 7f0e7e200640 1 -- 172.21.15.138:0/814472999 wait complete. 2024-01-21T22:25:24.601 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542155 for osd.4 2024-01-21T22:25:24.602 DEBUG:teuthology.parallel:result is None 2024-01-21T22:25:24.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.627+0000 7fd968c00640 1 -- 172.21.15.138:0/2202753175 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fd9641e5a00 con 0x7fd9640ca7b0 2024-01-21T22:25:24.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.631+0000 7fd95b400640 1 -- 172.21.15.138:0/2202753175 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd95407b9c0 con 0x7fd9640ca7b0 2024-01-21T22:25:24.633 INFO:teuthology.orchestra.run.smithi138.stdout:60129542156 2024-01-21T22:25:24.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.634+0000 7fd968c00640 1 -- 172.21.15.138:0/2202753175 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd93c06b360 msgr2=0x7fd93c06d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.634+0000 7fd968c00640 1 --2- 172.21.15.138:0/2202753175 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd93c06b360 0x7fd93c06d820 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd9641573a0 tx=0x7fd94c009040 comp rx=0 tx=0).stop 2024-01-21T22:25:24.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.634+0000 7fd968c00640 1 -- 172.21.15.138:0/2202753175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9640ca7b0 msgr2=0x7fd9641f2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.634+0000 7fd968c00640 1 --2- 172.21.15.138:0/2202753175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9640ca7b0 0x7fd9641f2af0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd954000c00 tx=0x7fd954004730 comp rx=0 tx=0).stop 2024-01-21T22:25:24.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.634+0000 7fd968c00640 1 -- 172.21.15.138:0/2202753175 shutdown_connections 2024-01-21T22:25:24.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.634+0000 7fd968c00640 1 --2- 172.21.15.138:0/2202753175 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd93c06b360 0x7fd93c06d820 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.634+0000 7fd968c00640 1 --2- 172.21.15.138:0/2202753175 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd96415d1a0 0x7fd9640c6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.634+0000 7fd968c00640 1 --2- 172.21.15.138:0/2202753175 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd96415cdc0 0x7fd9641576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.634+0000 7fd968c00640 1 --2- 172.21.15.138:0/2202753175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd9640ca7b0 0x7fd9641f2af0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.634+0000 7fd968c00640 1 -- 172.21.15.138:0/2202753175 >> 172.21.15.138:0/2202753175 conn(0x7fd96406d370 msgr2=0x7fd9640cd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:24.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.634+0000 7fd968c00640 1 -- 172.21.15.138:0/2202753175 shutdown_connections 2024-01-21T22:25:24.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.634+0000 7fd968c00640 1 -- 172.21.15.138:0/2202753175 wait complete. 2024-01-21T22:25:24.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.636+0000 7fd8c6600640 1 -- 172.21.15.138:0/3416329108 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fd8c004ea30 con 0x7fd8c006c9b0 2024-01-21T22:25:24.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.640+0000 7fd8b7400640 1 -- 172.21.15.138:0/3416329108 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd8a8034090 con 0x7fd8c006c9b0 2024-01-21T22:25:24.642 INFO:teuthology.orchestra.run.smithi138.stdout:60129542156 2024-01-21T22:25:24.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.642+0000 7fd8c6600640 1 -- 172.21.15.138:0/3416329108 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd89806b3b0 msgr2=0x7fd89806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.642+0000 7fd8c6600640 1 --2- 172.21.15.138:0/3416329108 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd89806b3b0 0x7fd89806d870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0009710 tx=0x7fd8b0006c90 comp rx=0 tx=0).stop 2024-01-21T22:25:24.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.642+0000 7fd8c6600640 1 -- 172.21.15.138:0/3416329108 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8c006c9b0 msgr2=0x7fd8c00caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.642+0000 7fd8c6600640 1 --2- 172.21.15.138:0/3416329108 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8c006c9b0 0x7fd8c00caa80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8000c00 tx=0x7fd8a8004730 comp rx=0 tx=0).stop 2024-01-21T22:25:24.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.643+0000 7fd8c6600640 1 -- 172.21.15.138:0/3416329108 shutdown_connections 2024-01-21T22:25:24.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.643+0000 7fd8c6600640 1 --2- 172.21.15.138:0/3416329108 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd89806b3b0 0x7fd89806d870 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.643+0000 7fd8c6600640 1 --2- 172.21.15.138:0/3416329108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c00c5970 0x7fd8c0071350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.643+0000 7fd8c6600640 1 --2- 172.21.15.138:0/3416329108 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd8c006c9b0 0x7fd8c00caa80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.643+0000 7fd8c6600640 1 --2- 172.21.15.138:0/3416329108 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd8c006c5d0 0x7fd8c00c5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.643+0000 7fd8c6600640 1 -- 172.21.15.138:0/3416329108 >> 172.21.15.138:0/3416329108 conn(0x7fd8c00cb070 msgr2=0x7fd8c006cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:24.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.643+0000 7fd8c6600640 1 -- 172.21.15.138:0/3416329108 shutdown_connections 2024-01-21T22:25:24.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.643+0000 7fd8c6600640 1 -- 172.21.15.138:0/3416329108 wait complete. 2024-01-21T22:25:24.657 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542156 for osd.6 2024-01-21T22:25:24.657 DEBUG:teuthology.parallel:result is None 2024-01-21T22:25:24.662 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542156 for osd.5 2024-01-21T22:25:24.662 DEBUG:teuthology.parallel:result is None 2024-01-21T22:25:24.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.772+0000 7fe98e800640 1 -- 172.21.15.138:0/3991074686 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe9880b4e50 con 0x7fe9880c6620 2024-01-21T22:25:24.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.777+0000 7fe97f400640 1 -- 172.21.15.138:0/3991074686 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe97807a870 con 0x7fe9880c6620 2024-01-21T22:25:24.779 INFO:teuthology.orchestra.run.smithi138.stdout:60129542154 2024-01-21T22:25:24.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.779+0000 7fe98e800640 1 -- 172.21.15.138:0/3991074686 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe96006b380 msgr2=0x7fe96006d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.779+0000 7fe98e800640 1 --2- 172.21.15.138:0/3991074686 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe96006b380 0x7fe96006d840 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe98806b2e0 tx=0x7fe970006c90 comp rx=0 tx=0).stop 2024-01-21T22:25:24.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.779+0000 7fe98e800640 1 -- 172.21.15.138:0/3991074686 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9880c6620 msgr2=0x7fe9880c2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:24.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.779+0000 7fe98e800640 1 --2- 172.21.15.138:0/3991074686 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9880c6620 0x7fe9880c2f20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe978009990 tx=0x7fe978004730 comp rx=0 tx=0).stop 2024-01-21T22:25:24.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.779+0000 7fe98e800640 1 -- 172.21.15.138:0/3991074686 shutdown_connections 2024-01-21T22:25:24.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.779+0000 7fe98e800640 1 --2- 172.21.15.138:0/3991074686 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe96006b380 0x7fe96006d840 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.779+0000 7fe98e800640 1 --2- 172.21.15.138:0/3991074686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe988153ef0 0x7fe988146de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.779+0000 7fe98e800640 1 --2- 172.21.15.138:0/3991074686 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe9880caf80 0x7fe9880c3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.779+0000 7fe98e800640 1 --2- 172.21.15.138:0/3991074686 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe9880c6620 0x7fe9880c2f20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:24.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.779+0000 7fe98e800640 1 -- 172.21.15.138:0/3991074686 >> 172.21.15.138:0/3991074686 conn(0x7fe9880cbca0 msgr2=0x7fe98806a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:24.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.779+0000 7fe98e800640 1 -- 172.21.15.138:0/3991074686 shutdown_connections 2024-01-21T22:25:24.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:24.779+0000 7fe98e800640 1 -- 172.21.15.138:0/3991074686 wait complete. 2024-01-21T22:25:24.798 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.2 2024-01-21T22:25:24.798 DEBUG:teuthology.parallel:result is None 2024-01-21T22:25:24.799 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-21T22:25:24.799 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-21T22:25:25.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.008+0000 7f8122e00640 1 Processor -- start 2024-01-21T22:25:25.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.008+0000 7f8122e00640 1 -- start start 2024-01-21T22:25:25.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.009+0000 7f8122e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f811c1504b0 0x7f811c150890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:25.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.009+0000 7f8122e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f811c157160 0x7f811c0c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:25.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.009+0000 7f8122e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811c072c50 con 0x7f811c0c02b0 2024-01-21T22:25:25.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.009+0000 7f8122e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f811c071ae0 con 0x7f811c1504b0 2024-01-21T22:25:25.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.009+0000 7f8122e00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811c156cd0 con 0x7f811c157160 2024-01-21T22:25:25.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.010+0000 7f8122400640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f811c0c02b0 0x7f811c0c1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:38046/0 (socket says 172.21.15.138:38046) 2024-01-21T22:25:25.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.010+0000 7f8122400640 1 -- 172.21.15.138:0/2874505728 learned_addr learned my addr 172.21.15.138:0/2874505728 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:25.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.010+0000 7f8121000640 1 --2- 172.21.15.138:0/2874505728 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f811c157160 0x7f811c0c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:25.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.010+0000 7f8121a00640 1 --2- 172.21.15.138:0/2874505728 >> v2:172.21.15.152:3300/0 conn(0x7f811c1504b0 0x7f811c150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:25.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.013+0000 7f8121000640 1 -- 172.21.15.138:0/2874505728 >> v2:172.21.15.152:3300/0 conn(0x7f811c1504b0 msgr2=0x7f811c150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:25.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.013+0000 7f8121000640 1 --2- 172.21.15.138:0/2874505728 >> v2:172.21.15.152:3300/0 conn(0x7f811c1504b0 0x7f811c150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:25.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.013+0000 7f8121000640 1 -- 172.21.15.138:0/2874505728 >> v1:172.21.15.138:6789/0 conn(0x7f811c0c02b0 legacy=0x7f811c0c1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:25.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.013+0000 7f811be00640 1 -- 172.21.15.138:0/2874505728 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2635183857 0 0) 0x7f811c072c50 con 0x7f811c0c02b0 2024-01-21T22:25:25.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.013+0000 7f8121000640 1 -- 172.21.15.138:0/2874505728 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811c0c5fb0 con 0x7f811c157160 2024-01-21T22:25:25.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.015+0000 7f8121000640 1 --2- 172.21.15.138:0/2874505728 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f811c157160 0x7f811c0c5a70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f81040097a0 tx=0x7f810402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cedabf71bb4d260 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:25.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.019+0000 7f811be00640 1 -- 172.21.15.138:0/2874505728 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8104031830 con 0x7f811c157160 2024-01-21T22:25:25.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.019+0000 7f811be00640 1 -- 172.21.15.138:0/2874505728 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8104031990 con 0x7f811c157160 2024-01-21T22:25:25.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.019+0000 7f811be00640 1 -- 172.21.15.138:0/2874505728 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8104031d20 con 0x7f811c157160 2024-01-21T22:25:25.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.019+0000 7f8122e00640 1 -- 172.21.15.138:0/2874505728 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f811c157160 msgr2=0x7f811c0c5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:25.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.019+0000 7f8122e00640 1 --2- 172.21.15.138:0/2874505728 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f811c157160 0x7f811c0c5a70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f81040097a0 tx=0x7f810402b8d0 comp rx=0 tx=0).stop 2024-01-21T22:25:25.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.019+0000 7f8122e00640 1 -- 172.21.15.138:0/2874505728 shutdown_connections 2024-01-21T22:25:25.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.019+0000 7f8122e00640 1 --2- 172.21.15.138:0/2874505728 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f811c157160 0x7f811c0c5a70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:25.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.019+0000 7f8122e00640 1 --2- 172.21.15.138:0/2874505728 >> v2:172.21.15.152:3300/0 conn(0x7f811c1504b0 0x7f811c150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:25.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.019+0000 7f8122e00640 1 -- 172.21.15.138:0/2874505728 >> 172.21.15.138:0/2874505728 conn(0x7f811c06dcb0 msgr2=0x7f811c06e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:25.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.019+0000 7f8122e00640 1 -- 172.21.15.138:0/2874505728 shutdown_connections 2024-01-21T22:25:25.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.019+0000 7f8122e00640 1 -- 172.21.15.138:0/2874505728 wait complete. 2024-01-21T22:25:25.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.021+0000 7f8122e00640 1 Processor -- start 2024-01-21T22:25:25.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.021+0000 7f8122e00640 1 -- start start 2024-01-21T22:25:25.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.022+0000 7f8122e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f811c0c02b0 0x7f811c1465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:25.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.022+0000 7f8122e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f811c1504b0 0x7f811c143a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:25.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.022+0000 7f8122e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f811c157160 0x7f811c143f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:25.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.022+0000 7f8122e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c142a40 con 0x7f811c1504b0 2024-01-21T22:25:25.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.022+0000 7f8122e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c1ef3f0 con 0x7f811c0c02b0 2024-01-21T22:25:25.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.022+0000 7f8122e00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811c1ef690 con 0x7f811c157160 2024-01-21T22:25:25.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.022+0000 7f8121000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f811c1504b0 0x7f811c143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:25.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.022+0000 7f8121a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f811c0c02b0 0x7f811c1465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:25.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.023+0000 7f8122400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f811c157160 0x7f811c143f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:25.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.023+0000 7f8121000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f811c1504b0 0x7f811c143a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53264/0 (socket says 172.21.15.138:53264) 2024-01-21T22:25:25.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.023+0000 7f8121000640 1 -- 172.21.15.138:0/431333527 learned_addr learned my addr 172.21.15.138:0/431333527 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:25.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.026+0000 7f8122400640 1 -- 172.21.15.138:0/431333527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f811c0c02b0 msgr2=0x7f811c1465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:25.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.026+0000 7f8122400640 1 --2- 172.21.15.138:0/431333527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f811c0c02b0 0x7f811c1465d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:25.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.026+0000 7f8122400640 1 -- 172.21.15.138:0/431333527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f811c1504b0 msgr2=0x7f811c143a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:25.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.026+0000 7f8122400640 1 --2- 172.21.15.138:0/431333527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f811c1504b0 0x7f811c143a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:25.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.026+0000 7f8122400640 1 -- 172.21.15.138:0/431333527 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8104009400 con 0x7f811c157160 2024-01-21T22:25:25.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.028+0000 7f8122400640 1 --2- 172.21.15.138:0/431333527 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f811c157160 0x7f811c143f60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f811400c750 tx=0x7f811400cc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:25.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.032+0000 7f8119600640 1 -- 172.21.15.138:0/431333527 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8114004d60 con 0x7f811c157160 2024-01-21T22:25:25.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.032+0000 7f8119600640 1 -- 172.21.15.138:0/431333527 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811400a640 con 0x7f811c157160 2024-01-21T22:25:25.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.032+0000 7f8119600640 1 -- 172.21.15.138:0/431333527 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811400aa30 con 0x7f811c157160 2024-01-21T22:25:25.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.032+0000 7f8122e00640 1 -- 172.21.15.138:0/431333527 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811c1ef970 con 0x7f811c157160 2024-01-21T22:25:25.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.032+0000 7f8122e00640 1 -- 172.21.15.138:0/431333527 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811c1eff30 con 0x7f811c157160 2024-01-21T22:25:25.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.033+0000 7f8122e00640 1 -- 172.21.15.138:0/431333527 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f811c067ce0 con 0x7f811c157160 2024-01-21T22:25:25.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.053+0000 7f8119600640 1 -- 172.21.15.138:0/431333527 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f811400ab90 con 0x7f811c157160 2024-01-21T22:25:25.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.054+0000 7f8119600640 1 --2- 172.21.15.138:0/431333527 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f80f406b380 0x7f80f406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:25.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.054+0000 7f8121a00640 1 --2- 172.21.15.138:0/431333527 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f80f406b380 0x7f80f406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:25.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.054+0000 7f8119600640 1 -- 172.21.15.138:0/431333527 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f811400ae40 con 0x7f811c157160 2024-01-21T22:25:25.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.055+0000 7f8121a00640 1 --2- 172.21.15.138:0/431333527 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f80f406b380 0x7f80f406d840 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f810c009b40 tx=0x7f810c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:25.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.296+0000 7f8119600640 1 -- 172.21.15.138:0/431333527 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f81140544e0 con 0x7f811c157160 2024-01-21T22:25:25.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.493+0000 7f8122e00640 1 -- 172.21.15.138:0/431333527 --> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f811c072970 con 0x7f80f406b380 2024-01-21T22:25:25.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.500+0000 7f8119600640 1 -- 172.21.15.138:0/431333527 <== mgr.4111 v2:172.21.15.152:6800/314625564 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264479 (secure 0 0 0) 0x7f810c011a60 con 0x7f80f406b380 2024-01-21T22:25:25.506 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:25:25.507 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-01-21T22:25:25.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.506+0000 7f8122e00640 1 -- 172.21.15.138:0/431333527 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f80f406b380 msgr2=0x7f80f406d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:25.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.506+0000 7f8122e00640 1 --2- 172.21.15.138:0/431333527 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f80f406b380 0x7f80f406d840 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f810c009b40 tx=0x7f810c009190 comp rx=0 tx=0).stop 2024-01-21T22:25:25.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.506+0000 7f8122e00640 1 -- 172.21.15.138:0/431333527 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f811c157160 msgr2=0x7f811c143f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:25.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.506+0000 7f8122e00640 1 --2- 172.21.15.138:0/431333527 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f811c157160 0x7f811c143f60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f811400c750 tx=0x7f811400cc10 comp rx=0 tx=0).stop 2024-01-21T22:25:25.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.506+0000 7f8122e00640 1 -- 172.21.15.138:0/431333527 shutdown_connections 2024-01-21T22:25:25.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.506+0000 7f8122e00640 1 --2- 172.21.15.138:0/431333527 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f80f406b380 0x7f80f406d840 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:25.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.506+0000 7f8122e00640 1 --2- 172.21.15.138:0/431333527 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f811c157160 0x7f811c143f60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:25.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.506+0000 7f8122e00640 1 --2- 172.21.15.138:0/431333527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f811c1504b0 0x7f811c143a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:25.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.506+0000 7f8122e00640 1 --2- 172.21.15.138:0/431333527 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f811c0c02b0 0x7f811c1465d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:25.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.506+0000 7f8122e00640 1 -- 172.21.15.138:0/431333527 >> 172.21.15.138:0/431333527 conn(0x7f811c06dcb0 msgr2=0x7f811c0c7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:25.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.506+0000 7f8122e00640 1 -- 172.21.15.138:0/431333527 shutdown_connections 2024-01-21T22:25:25.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.506+0000 7f8122e00640 1 -- 172.21.15.138:0/431333527 wait complete. 2024-01-21T22:25:25.524 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-01-21T22:25:25.219307+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219240,"kb_used_data":3080,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755480,"statfs":{"total":773094113280,"available":772869611520,"internally_reserved":0,"allocated":3153920,"data_stored":1865780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002962"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286921+0000","last_change":"2024-01-21T22:25:04.888366+0000","last_active":"2024-01-21T22:25:07.286921+0000","last_peered":"2024-01-21T22:25:07.286921+0000","last_clean":"2024-01-21T22:25:07.286921+0000","last_became_active":"2024-01-21T22:25:04.888060+0000","last_became_peered":"2024-01-21T22:25:04.888060+0000","last_unstale":"2024-01-21T22:25:07.286921+0000","last_undegraded":"2024-01-21T22:25:07.286921+0000","last_fullsized":"2024-01-21T22:25:07.286921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:32:13.512213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298177+0000","last_change":"2024-01-21T22:25:04.843704+0000","last_active":"2024-01-21T22:25:07.298177+0000","last_peered":"2024-01-21T22:25:07.298177+0000","last_clean":"2024-01-21T22:25:07.298177+0000","last_became_active":"2024-01-21T22:25:04.843471+0000","last_became_peered":"2024-01-21T22:25:04.843471+0000","last_unstale":"2024-01-21T22:25:07.298177+0000","last_undegraded":"2024-01-21T22:25:07.298177+0000","last_fullsized":"2024-01-21T22:25:07.298177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:34:24.822768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298055+0000","last_change":"2024-01-21T22:25:04.858021+0000","last_active":"2024-01-21T22:25:07.298055+0000","last_peered":"2024-01-21T22:25:07.298055+0000","last_clean":"2024-01-21T22:25:07.298055+0000","last_became_active":"2024-01-21T22:25:04.857685+0000","last_became_peered":"2024-01-21T22:25:04.857685+0000","last_unstale":"2024-01-21T22:25:07.298055+0000","last_undegraded":"2024-01-21T22:25:07.298055+0000","last_fullsized":"2024-01-21T22:25:07.298055+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:29:02.950903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.144191+0000","last_change":"2024-01-21T22:25:04.858941+0000","last_active":"2024-01-21T22:25:07.144191+0000","last_peered":"2024-01-21T22:25:07.144191+0000","last_clean":"2024-01-21T22:25:07.144191+0000","last_became_active":"2024-01-21T22:25:04.858278+0000","last_became_peered":"2024-01-21T22:25:04.858278+0000","last_unstale":"2024-01-21T22:25:07.144191+0000","last_undegraded":"2024-01-21T22:25:07.144191+0000","last_fullsized":"2024-01-21T22:25:07.144191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T10:03:50.093635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286258+0000","last_change":"2024-01-21T22:25:04.881014+0000","last_active":"2024-01-21T22:25:07.286258+0000","last_peered":"2024-01-21T22:25:07.286258+0000","last_clean":"2024-01-21T22:25:07.286258+0000","last_became_active":"2024-01-21T22:25:04.878457+0000","last_became_peered":"2024-01-21T22:25:04.878457+0000","last_unstale":"2024-01-21T22:25:07.286258+0000","last_undegraded":"2024-01-21T22:25:07.286258+0000","last_fullsized":"2024-01-21T22:25:07.286258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:34:24.822768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.143791+0000","last_change":"2024-01-21T22:25:04.859776+0000","last_active":"2024-01-21T22:25:07.143791+0000","last_peered":"2024-01-21T22:25:07.143791+0000","last_clean":"2024-01-21T22:25:07.143791+0000","last_became_active":"2024-01-21T22:25:04.858731+0000","last_became_peered":"2024-01-21T22:25:04.858731+0000","last_unstale":"2024-01-21T22:25:07.143791+0000","last_undegraded":"2024-01-21T22:25:07.143791+0000","last_fullsized":"2024-01-21T22:25:07.143791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:01:30.091588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.285525+0000","last_change":"2024-01-21T22:25:04.881392+0000","last_active":"2024-01-21T22:25:07.285525+0000","last_peered":"2024-01-21T22:25:07.285525+0000","last_clean":"2024-01-21T22:25:07.285525+0000","last_became_active":"2024-01-21T22:25:04.881090+0000","last_became_peered":"2024-01-21T22:25:04.881090+0000","last_unstale":"2024-01-21T22:25:07.285525+0000","last_undegraded":"2024-01-21T22:25:07.285525+0000","last_fullsized":"2024-01-21T22:25:07.285525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:25:50.380424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297019+0000","last_change":"2024-01-21T22:25:04.879186+0000","last_active":"2024-01-21T22:25:07.297019+0000","last_peered":"2024-01-21T22:25:07.297019+0000","last_clean":"2024-01-21T22:25:07.297019+0000","last_became_active":"2024-01-21T22:25:04.878824+0000","last_became_peered":"2024-01-21T22:25:04.878824+0000","last_unstale":"2024-01-21T22:25:07.297019+0000","last_undegraded":"2024-01-21T22:25:07.297019+0000","last_fullsized":"2024-01-21T22:25:07.297019+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:31:25.985811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304238+0000","last_change":"2024-01-21T22:25:04.886391+0000","last_active":"2024-01-21T22:25:07.304238+0000","last_peered":"2024-01-21T22:25:07.304238+0000","last_clean":"2024-01-21T22:25:07.304238+0000","last_became_active":"2024-01-21T22:25:04.884979+0000","last_became_peered":"2024-01-21T22:25:04.884979+0000","last_unstale":"2024-01-21T22:25:07.304238+0000","last_undegraded":"2024-01-21T22:25:07.304238+0000","last_fullsized":"2024-01-21T22:25:07.304238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:36:36.009632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286546+0000","last_change":"2024-01-21T22:25:04.888126+0000","last_active":"2024-01-21T22:25:07.286546+0000","last_peered":"2024-01-21T22:25:07.286546+0000","last_clean":"2024-01-21T22:25:07.286546+0000","last_became_active":"2024-01-21T22:25:04.887631+0000","last_became_peered":"2024-01-21T22:25:04.887631+0000","last_unstale":"2024-01-21T22:25:07.286546+0000","last_undegraded":"2024-01-21T22:25:07.286546+0000","last_fullsized":"2024-01-21T22:25:07.286546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:00:02.684745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286056+0000","last_change":"2024-01-21T22:25:04.885677+0000","last_active":"2024-01-21T22:25:07.286056+0000","last_peered":"2024-01-21T22:25:07.286056+0000","last_clean":"2024-01-21T22:25:07.286056+0000","last_became_active":"2024-01-21T22:25:04.884697+0000","last_became_peered":"2024-01-21T22:25:04.884697+0000","last_unstale":"2024-01-21T22:25:07.286056+0000","last_undegraded":"2024-01-21T22:25:07.286056+0000","last_fullsized":"2024-01-21T22:25:07.286056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:44:23.749631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286124+0000","last_change":"2024-01-21T22:25:04.886799+0000","last_active":"2024-01-21T22:25:07.286124+0000","last_peered":"2024-01-21T22:25:07.286124+0000","last_clean":"2024-01-21T22:25:07.286124+0000","last_became_active":"2024-01-21T22:25:04.886387+0000","last_became_peered":"2024-01-21T22:25:04.886387+0000","last_unstale":"2024-01-21T22:25:07.286124+0000","last_undegraded":"2024-01-21T22:25:07.286124+0000","last_fullsized":"2024-01-21T22:25:07.286124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:42:39.977125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304238+0000","last_change":"2024-01-21T22:25:04.888103+0000","last_active":"2024-01-21T22:25:07.304238+0000","last_peered":"2024-01-21T22:25:07.304238+0000","last_clean":"2024-01-21T22:25:07.304238+0000","last_became_active":"2024-01-21T22:25:04.887908+0000","last_became_peered":"2024-01-21T22:25:04.887908+0000","last_unstale":"2024-01-21T22:25:07.304238+0000","last_undegraded":"2024-01-21T22:25:07.304238+0000","last_fullsized":"2024-01-21T22:25:07.304238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:23:52.444277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304247+0000","last_change":"2024-01-21T22:25:04.842750+0000","last_active":"2024-01-21T22:25:07.304247+0000","last_peered":"2024-01-21T22:25:07.304247+0000","last_clean":"2024-01-21T22:25:07.304247+0000","last_became_active":"2024-01-21T22:25:04.842512+0000","last_became_peered":"2024-01-21T22:25:04.842512+0000","last_unstale":"2024-01-21T22:25:07.304247+0000","last_undegraded":"2024-01-21T22:25:07.304247+0000","last_fullsized":"2024-01-21T22:25:07.304247+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:40:05.554704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.296486+0000","last_change":"2024-01-21T22:25:04.876592+0000","last_active":"2024-01-21T22:25:07.296486+0000","last_peered":"2024-01-21T22:25:07.296486+0000","last_clean":"2024-01-21T22:25:07.296486+0000","last_became_active":"2024-01-21T22:25:04.876305+0000","last_became_peered":"2024-01-21T22:25:04.876305+0000","last_unstale":"2024-01-21T22:25:07.296486+0000","last_undegraded":"2024-01-21T22:25:07.296486+0000","last_fullsized":"2024-01-21T22:25:07.296486+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:32.177562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093349+0000","last_change":"2024-01-21T22:25:04.878038+0000","last_active":"2024-01-21T22:25:07.093349+0000","last_peered":"2024-01-21T22:25:07.093349+0000","last_clean":"2024-01-21T22:25:07.093349+0000","last_became_active":"2024-01-21T22:25:04.877716+0000","last_became_peered":"2024-01-21T22:25:04.877716+0000","last_unstale":"2024-01-21T22:25:07.093349+0000","last_undegraded":"2024-01-21T22:25:07.093349+0000","last_fullsized":"2024-01-21T22:25:07.093349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:32:13.512213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.094445+0000","last_change":"2024-01-21T22:25:04.890537+0000","last_active":"2024-01-21T22:25:07.094445+0000","last_peered":"2024-01-21T22:25:07.094445+0000","last_clean":"2024-01-21T22:25:07.094445+0000","last_became_active":"2024-01-21T22:25:04.890298+0000","last_became_peered":"2024-01-21T22:25:04.890298+0000","last_unstale":"2024-01-21T22:25:07.094445+0000","last_undegraded":"2024-01-21T22:25:07.094445+0000","last_fullsized":"2024-01-21T22:25:07.094445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:23:57.435028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.296659+0000","last_change":"2024-01-21T22:25:04.712868+0000","last_active":"2024-01-21T22:25:07.296659+0000","last_peered":"2024-01-21T22:25:07.296659+0000","last_clean":"2024-01-21T22:25:07.296659+0000","last_became_active":"2024-01-21T22:25:04.712616+0000","last_became_peered":"2024-01-21T22:25:04.712616+0000","last_unstale":"2024-01-21T22:25:07.296659+0000","last_undegraded":"2024-01-21T22:25:07.296659+0000","last_fullsized":"2024-01-21T22:25:07.296659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:50:41.306582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.094344+0000","last_change":"2024-01-21T22:25:04.879328+0000","last_active":"2024-01-21T22:25:07.094344+0000","last_peered":"2024-01-21T22:25:07.094344+0000","last_clean":"2024-01-21T22:25:07.094344+0000","last_became_active":"2024-01-21T22:25:04.879011+0000","last_became_peered":"2024-01-21T22:25:04.879011+0000","last_unstale":"2024-01-21T22:25:07.094344+0000","last_undegraded":"2024-01-21T22:25:07.094344+0000","last_fullsized":"2024-01-21T22:25:07.094344+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:31:23.569672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286286+0000","last_change":"2024-01-21T22:25:04.885620+0000","last_active":"2024-01-21T22:25:07.286286+0000","last_peered":"2024-01-21T22:25:07.286286+0000","last_clean":"2024-01-21T22:25:07.286286+0000","last_became_active":"2024-01-21T22:25:04.884364+0000","last_became_peered":"2024-01-21T22:25:04.884364+0000","last_unstale":"2024-01-21T22:25:07.286286+0000","last_undegraded":"2024-01-21T22:25:07.286286+0000","last_fullsized":"2024-01-21T22:25:07.286286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:13:04.601911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151781+0000","last_change":"2024-01-21T22:25:04.868371+0000","last_active":"2024-01-21T22:25:07.151781+0000","last_peered":"2024-01-21T22:25:07.151781+0000","last_clean":"2024-01-21T22:25:07.151781+0000","last_became_active":"2024-01-21T22:25:04.868196+0000","last_became_peered":"2024-01-21T22:25:04.868196+0000","last_unstale":"2024-01-21T22:25:07.151781+0000","last_undegraded":"2024-01-21T22:25:07.151781+0000","last_fullsized":"2024-01-21T22:25:07.151781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:05:41.950232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.303520+0000","last_change":"2024-01-21T22:25:04.879144+0000","last_active":"2024-01-21T22:25:07.303520+0000","last_peered":"2024-01-21T22:25:07.303520+0000","last_clean":"2024-01-21T22:25:07.303520+0000","last_became_active":"2024-01-21T22:25:04.878868+0000","last_became_peered":"2024-01-21T22:25:04.878868+0000","last_unstale":"2024-01-21T22:25:07.303520+0000","last_undegraded":"2024-01-21T22:25:07.303520+0000","last_fullsized":"2024-01-21T22:25:07.303520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:55:28.069411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.296623+0000","last_change":"2024-01-21T22:25:04.715246+0000","last_active":"2024-01-21T22:25:07.296623+0000","last_peered":"2024-01-21T22:25:07.296623+0000","last_clean":"2024-01-21T22:25:07.296623+0000","last_became_active":"2024-01-21T22:25:04.714969+0000","last_became_peered":"2024-01-21T22:25:04.714969+0000","last_unstale":"2024-01-21T22:25:07.296623+0000","last_undegraded":"2024-01-21T22:25:07.296623+0000","last_fullsized":"2024-01-21T22:25:07.296623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:56:24.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093802+0000","last_change":"2024-01-21T22:25:04.877902+0000","last_active":"2024-01-21T22:25:07.093802+0000","last_peered":"2024-01-21T22:25:07.093802+0000","last_clean":"2024-01-21T22:25:07.093802+0000","last_became_active":"2024-01-21T22:25:04.876963+0000","last_became_peered":"2024-01-21T22:25:04.876963+0000","last_unstale":"2024-01-21T22:25:07.093802+0000","last_undegraded":"2024-01-21T22:25:07.093802+0000","last_fullsized":"2024-01-21T22:25:07.093802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:30:48.864454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297800+0000","last_change":"2024-01-21T22:25:04.716735+0000","last_active":"2024-01-21T22:25:07.297800+0000","last_peered":"2024-01-21T22:25:07.297800+0000","last_clean":"2024-01-21T22:25:07.297800+0000","last_became_active":"2024-01-21T22:25:04.716570+0000","last_became_peered":"2024-01-21T22:25:04.716570+0000","last_unstale":"2024-01-21T22:25:07.297800+0000","last_undegraded":"2024-01-21T22:25:07.297800+0000","last_fullsized":"2024-01-21T22:25:07.297800+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:44:55.158990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.143839+0000","last_change":"2024-01-21T22:25:04.886230+0000","last_active":"2024-01-21T22:25:07.143839+0000","last_peered":"2024-01-21T22:25:07.143839+0000","last_clean":"2024-01-21T22:25:07.143839+0000","last_became_active":"2024-01-21T22:25:04.885523+0000","last_became_peered":"2024-01-21T22:25:04.885523+0000","last_unstale":"2024-01-21T22:25:07.143839+0000","last_undegraded":"2024-01-21T22:25:07.143839+0000","last_fullsized":"2024-01-21T22:25:07.143839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:06:02.934920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304425+0000","last_change":"2024-01-21T22:25:04.842692+0000","last_active":"2024-01-21T22:25:07.304425+0000","last_peered":"2024-01-21T22:25:07.304425+0000","last_clean":"2024-01-21T22:25:07.304425+0000","last_became_active":"2024-01-21T22:25:04.842429+0000","last_became_peered":"2024-01-21T22:25:04.842429+0000","last_unstale":"2024-01-21T22:25:07.304425+0000","last_undegraded":"2024-01-21T22:25:07.304425+0000","last_fullsized":"2024-01-21T22:25:07.304425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:52:24.233709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304314+0000","last_change":"2024-01-21T22:25:04.888296+0000","last_active":"2024-01-21T22:25:07.304314+0000","last_peered":"2024-01-21T22:25:07.304314+0000","last_clean":"2024-01-21T22:25:07.304314+0000","last_became_active":"2024-01-21T22:25:04.888025+0000","last_became_peered":"2024-01-21T22:25:04.888025+0000","last_unstale":"2024-01-21T22:25:07.304314+0000","last_undegraded":"2024-01-21T22:25:07.304314+0000","last_fullsized":"2024-01-21T22:25:07.304314+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:57:44.129490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297099+0000","last_change":"2024-01-21T22:25:04.711908+0000","last_active":"2024-01-21T22:25:07.297099+0000","last_peered":"2024-01-21T22:25:07.297099+0000","last_clean":"2024-01-21T22:25:07.297099+0000","last_became_active":"2024-01-21T22:25:04.711654+0000","last_became_peered":"2024-01-21T22:25:04.711654+0000","last_unstale":"2024-01-21T22:25:07.297099+0000","last_undegraded":"2024-01-21T22:25:07.297099+0000","last_fullsized":"2024-01-21T22:25:07.297099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:20:41.612265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093463+0000","last_change":"2024-01-21T22:25:04.876757+0000","last_active":"2024-01-21T22:25:07.093463+0000","last_peered":"2024-01-21T22:25:07.093463+0000","last_clean":"2024-01-21T22:25:07.093463+0000","last_became_active":"2024-01-21T22:25:04.876012+0000","last_became_peered":"2024-01-21T22:25:04.876012+0000","last_unstale":"2024-01-21T22:25:07.093463+0000","last_undegraded":"2024-01-21T22:25:07.093463+0000","last_fullsized":"2024-01-21T22:25:07.093463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:00:56.239773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.296179+0000","last_change":"2024-01-21T22:25:04.715277+0000","last_active":"2024-01-21T22:25:07.296179+0000","last_peered":"2024-01-21T22:25:07.296179+0000","last_clean":"2024-01-21T22:25:07.296179+0000","last_became_active":"2024-01-21T22:25:04.715034+0000","last_became_peered":"2024-01-21T22:25:04.715034+0000","last_unstale":"2024-01-21T22:25:07.296179+0000","last_undegraded":"2024-01-21T22:25:07.296179+0000","last_fullsized":"2024-01-21T22:25:07.296179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:55:28.069411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.143324+0000","last_change":"2024-01-21T22:25:04.844472+0000","last_active":"2024-01-21T22:25:07.143324+0000","last_peered":"2024-01-21T22:25:07.143324+0000","last_clean":"2024-01-21T22:25:07.143324+0000","last_became_active":"2024-01-21T22:25:04.843827+0000","last_became_peered":"2024-01-21T22:25:04.843827+0000","last_unstale":"2024-01-21T22:25:07.143324+0000","last_undegraded":"2024-01-21T22:25:07.143324+0000","last_fullsized":"2024-01-21T22:25:07.143324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:58:10.552987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.303755+0000","last_change":"2024-01-21T22:25:04.713764+0000","last_active":"2024-01-21T22:25:07.303755+0000","last_peered":"2024-01-21T22:25:07.303755+0000","last_clean":"2024-01-21T22:25:07.303755+0000","last_became_active":"2024-01-21T22:25:04.712814+0000","last_became_peered":"2024-01-21T22:25:04.712814+0000","last_unstale":"2024-01-21T22:25:07.303755+0000","last_undegraded":"2024-01-21T22:25:07.303755+0000","last_fullsized":"2024-01-21T22:25:07.303755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:16:41.440927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297178+0000","last_change":"2024-01-21T22:25:03.703079+0000","last_active":"2024-01-21T22:25:07.297178+0000","last_peered":"2024-01-21T22:25:07.297178+0000","last_clean":"2024-01-21T22:25:07.297178+0000","last_became_active":"2024-01-21T22:25:03.702776+0000","last_became_peered":"2024-01-21T22:25:03.702776+0000","last_unstale":"2024-01-21T22:25:07.297178+0000","last_undegraded":"2024-01-21T22:25:07.297178+0000","last_fullsized":"2024-01-21T22:25:07.297178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:34:13.302149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151769+0000","last_change":"2024-01-21T22:25:04.877158+0000","last_active":"2024-01-21T22:25:07.151769+0000","last_peered":"2024-01-21T22:25:07.151769+0000","last_clean":"2024-01-21T22:25:07.151769+0000","last_became_active":"2024-01-21T22:25:04.876920+0000","last_became_peered":"2024-01-21T22:25:04.876920+0000","last_unstale":"2024-01-21T22:25:07.151769+0000","last_undegraded":"2024-01-21T22:25:07.151769+0000","last_fullsized":"2024-01-21T22:25:07.151769+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:46:55.451173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093458+0000","last_change":"2024-01-21T22:25:03.720178+0000","last_active":"2024-01-21T22:25:07.093458+0000","last_peered":"2024-01-21T22:25:07.093458+0000","last_clean":"2024-01-21T22:25:07.093458+0000","last_became_active":"2024-01-21T22:25:03.719909+0000","last_became_peered":"2024-01-21T22:25:03.719909+0000","last_unstale":"2024-01-21T22:25:07.093458+0000","last_undegraded":"2024-01-21T22:25:07.093458+0000","last_fullsized":"2024-01-21T22:25:07.093458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:44:53.910872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.877462+0000","last_change":"2024-01-21T22:25:03.723005+0000","last_active":"2024-01-21T22:25:05.877462+0000","last_peered":"2024-01-21T22:25:05.877462+0000","last_clean":"2024-01-21T22:25:05.877462+0000","last_became_active":"2024-01-21T22:25:03.722758+0000","last_became_peered":"2024-01-21T22:25:03.722758+0000","last_unstale":"2024-01-21T22:25:05.877462+0000","last_undegraded":"2024-01-21T22:25:05.877462+0000","last_fullsized":"2024-01-21T22:25:05.877462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:16:40.192808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297215+0000","last_change":"2024-01-21T22:25:04.844030+0000","last_active":"2024-01-21T22:25:07.297215+0000","last_peered":"2024-01-21T22:25:07.297215+0000","last_clean":"2024-01-21T22:25:07.297215+0000","last_became_active":"2024-01-21T22:25:04.843714+0000","last_became_peered":"2024-01-21T22:25:04.843714+0000","last_unstale":"2024-01-21T22:25:07.297215+0000","last_undegraded":"2024-01-21T22:25:07.297215+0000","last_fullsized":"2024-01-21T22:25:07.297215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:25:50.380424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.521757+0000","last_change":"2024-01-21T22:25:03.702767+0000","last_active":"2024-01-21T22:25:05.521757+0000","last_peered":"2024-01-21T22:25:05.521757+0000","last_clean":"2024-01-21T22:25:05.521757+0000","last_became_active":"2024-01-21T22:25:03.702140+0000","last_became_peered":"2024-01-21T22:25:03.702140+0000","last_unstale":"2024-01-21T22:25:05.521757+0000","last_undegraded":"2024-01-21T22:25:05.521757+0000","last_fullsized":"2024-01-21T22:25:05.521757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:46:54.203055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151688+0000","last_change":"2024-01-21T22:25:04.867173+0000","last_active":"2024-01-21T22:25:07.151688+0000","last_peered":"2024-01-21T22:25:07.151688+0000","last_clean":"2024-01-21T22:25:07.151688+0000","last_became_active":"2024-01-21T22:25:04.866945+0000","last_became_peered":"2024-01-21T22:25:04.866945+0000","last_unstale":"2024-01-21T22:25:07.151688+0000","last_undegraded":"2024-01-21T22:25:07.151688+0000","last_fullsized":"2024-01-21T22:25:07.151688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:15:41.725132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.882129+0000","last_change":"2024-01-21T22:25:03.715128+0000","last_active":"2024-01-21T22:25:05.882129+0000","last_peered":"2024-01-21T22:25:05.882129+0000","last_clean":"2024-01-21T22:25:05.882129+0000","last_became_active":"2024-01-21T22:25:03.714039+0000","last_became_peered":"2024-01-21T22:25:03.714039+0000","last_unstale":"2024-01-21T22:25:05.882129+0000","last_undegraded":"2024-01-21T22:25:05.882129+0000","last_fullsized":"2024-01-21T22:25:05.882129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:09:09.988399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.144063+0000","last_change":"2024-01-21T22:25:04.886534+0000","last_active":"2024-01-21T22:25:07.144063+0000","last_peered":"2024-01-21T22:25:07.144063+0000","last_clean":"2024-01-21T22:25:07.144063+0000","last_became_active":"2024-01-21T22:25:04.885963+0000","last_became_peered":"2024-01-21T22:25:04.885963+0000","last_unstale":"2024-01-21T22:25:07.144063+0000","last_undegraded":"2024-01-21T22:25:07.144063+0000","last_fullsized":"2024-01-21T22:25:07.144063+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:44:31.867220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.610812+0000","last_change":"2024-01-21T22:25:03.718237+0000","last_active":"2024-01-21T22:25:05.610812+0000","last_peered":"2024-01-21T22:25:05.610812+0000","last_clean":"2024-01-21T22:25:05.610812+0000","last_became_active":"2024-01-21T22:25:03.717937+0000","last_became_peered":"2024-01-21T22:25:03.717937+0000","last_unstale":"2024-01-21T22:25:05.610812+0000","last_undegraded":"2024-01-21T22:25:05.610812+0000","last_fullsized":"2024-01-21T22:25:05.610812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:31:24.737692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298047+0000","last_change":"2024-01-21T22:25:04.882743+0000","last_active":"2024-01-21T22:25:07.298047+0000","last_peered":"2024-01-21T22:25:07.298047+0000","last_clean":"2024-01-21T22:25:07.298047+0000","last_became_active":"2024-01-21T22:25:04.882478+0000","last_became_peered":"2024-01-21T22:25:04.882478+0000","last_unstale":"2024-01-21T22:25:07.298047+0000","last_undegraded":"2024-01-21T22:25:07.298047+0000","last_fullsized":"2024-01-21T22:25:07.298047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:42:39.977125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.146198+0000","last_change":"2024-01-21T22:25:03.703672+0000","last_active":"2024-01-21T22:25:07.146198+0000","last_peered":"2024-01-21T22:25:07.146198+0000","last_clean":"2024-01-21T22:25:07.146198+0000","last_became_active":"2024-01-21T22:25:03.703426+0000","last_became_peered":"2024-01-21T22:25:03.703426+0000","last_unstale":"2024-01-21T22:25:07.146198+0000","last_undegraded":"2024-01-21T22:25:07.146198+0000","last_fullsized":"2024-01-21T22:25:07.146198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:41:51.793206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093763+0000","last_change":"2024-01-21T22:25:04.887306+0000","last_active":"2024-01-21T22:25:07.093763+0000","last_peered":"2024-01-21T22:25:07.093763+0000","last_clean":"2024-01-21T22:25:07.093763+0000","last_became_active":"2024-01-21T22:25:04.886970+0000","last_became_peered":"2024-01-21T22:25:04.886970+0000","last_unstale":"2024-01-21T22:25:07.093763+0000","last_undegraded":"2024-01-21T22:25:07.093763+0000","last_fullsized":"2024-01-21T22:25:07.093763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:59:45.239426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"23'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.158991+0000","last_change":"2024-01-21T22:25:03.714102+0000","last_active":"2024-01-21T22:25:07.158991+0000","last_peered":"2024-01-21T22:25:07.158991+0000","last_clean":"2024-01-21T22:25:07.158991+0000","last_became_active":"2024-01-21T22:25:03.713717+0000","last_became_peered":"2024-01-21T22:25:03.713717+0000","last_unstale":"2024-01-21T22:25:07.158991+0000","last_undegraded":"2024-01-21T22:25:07.158991+0000","last_fullsized":"2024-01-21T22:25:07.158991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:32:15.181238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286154+0000","last_change":"2024-01-21T22:25:04.881307+0000","last_active":"2024-01-21T22:25:07.286154+0000","last_peered":"2024-01-21T22:25:07.286154+0000","last_clean":"2024-01-21T22:25:07.286154+0000","last_became_active":"2024-01-21T22:25:04.880472+0000","last_became_peered":"2024-01-21T22:25:04.880472+0000","last_unstale":"2024-01-21T22:25:07.286154+0000","last_undegraded":"2024-01-21T22:25:07.286154+0000","last_fullsized":"2024-01-21T22:25:07.286154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:29:02.950903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151339+0000","last_change":"2024-01-21T22:25:03.721825+0000","last_active":"2024-01-21T22:25:07.151339+0000","last_peered":"2024-01-21T22:25:07.151339+0000","last_clean":"2024-01-21T22:25:07.151339+0000","last_became_active":"2024-01-21T22:25:03.721502+0000","last_became_peered":"2024-01-21T22:25:03.721502+0000","last_unstale":"2024-01-21T22:25:07.151339+0000","last_undegraded":"2024-01-21T22:25:07.151339+0000","last_fullsized":"2024-01-21T22:25:07.151339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:09:09.988399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304218+0000","last_change":"2024-01-21T22:25:04.877455+0000","last_active":"2024-01-21T22:25:07.304218+0000","last_peered":"2024-01-21T22:25:07.304218+0000","last_clean":"2024-01-21T22:25:07.304218+0000","last_became_active":"2024-01-21T22:25:04.877257+0000","last_became_peered":"2024-01-21T22:25:04.877257+0000","last_unstale":"2024-01-21T22:25:07.304218+0000","last_undegraded":"2024-01-21T22:25:07.304218+0000","last_fullsized":"2024-01-21T22:25:07.304218+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:34:14.550267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.522179+0000","last_change":"2024-01-21T22:25:03.706077+0000","last_active":"2024-01-21T22:25:05.522179+0000","last_peered":"2024-01-21T22:25:05.522179+0000","last_clean":"2024-01-21T22:25:05.522179+0000","last_became_active":"2024-01-21T22:25:03.702798+0000","last_became_peered":"2024-01-21T22:25:03.702798+0000","last_unstale":"2024-01-21T22:25:05.522179+0000","last_undegraded":"2024-01-21T22:25:05.522179+0000","last_fullsized":"2024-01-21T22:25:05.522179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:20:40.364147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093643+0000","last_change":"2024-01-21T22:25:04.890636+0000","last_active":"2024-01-21T22:25:07.093643+0000","last_peered":"2024-01-21T22:25:07.093643+0000","last_clean":"2024-01-21T22:25:07.093643+0000","last_became_active":"2024-01-21T22:25:04.890271+0000","last_became_peered":"2024-01-21T22:25:04.890271+0000","last_unstale":"2024-01-21T22:25:07.093643+0000","last_undegraded":"2024-01-21T22:25:07.093643+0000","last_fullsized":"2024-01-21T22:25:07.093643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:15:46.754395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.609220+0000","last_change":"2024-01-21T22:25:03.717202+0000","last_active":"2024-01-21T22:25:05.609220+0000","last_peered":"2024-01-21T22:25:05.609220+0000","last_clean":"2024-01-21T22:25:05.609220+0000","last_became_active":"2024-01-21T22:25:03.716846+0000","last_became_peered":"2024-01-21T22:25:03.716846+0000","last_unstale":"2024-01-21T22:25:05.609220+0000","last_undegraded":"2024-01-21T22:25:05.609220+0000","last_fullsized":"2024-01-21T22:25:05.609220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:10.904923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.303888+0000","last_change":"2024-01-21T22:25:04.715053+0000","last_active":"2024-01-21T22:25:07.303888+0000","last_peered":"2024-01-21T22:25:07.303888+0000","last_clean":"2024-01-21T22:25:07.303888+0000","last_became_active":"2024-01-21T22:25:04.713566+0000","last_became_peered":"2024-01-21T22:25:04.713566+0000","last_unstale":"2024-01-21T22:25:07.303888+0000","last_undegraded":"2024-01-21T22:25:07.303888+0000","last_fullsized":"2024-01-21T22:25:07.303888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:50:41.306582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"23'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304071+0000","last_change":"2024-01-21T22:25:03.703017+0000","last_active":"2024-01-21T22:25:07.304071+0000","last_peered":"2024-01-21T22:25:07.304071+0000","last_clean":"2024-01-21T22:25:07.304071+0000","last_became_active":"2024-01-21T22:25:03.702742+0000","last_became_peered":"2024-01-21T22:25:03.702742+0000","last_unstale":"2024-01-21T22:25:07.304071+0000","last_undegraded":"2024-01-21T22:25:07.304071+0000","last_fullsized":"2024-01-21T22:25:07.304071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:42:03.386594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093975+0000","last_change":"2024-01-21T22:25:04.876507+0000","last_active":"2024-01-21T22:25:07.093975+0000","last_peered":"2024-01-21T22:25:07.093975+0000","last_clean":"2024-01-21T22:25:07.093975+0000","last_became_active":"2024-01-21T22:25:04.876169+0000","last_became_peered":"2024-01-21T22:25:04.876169+0000","last_unstale":"2024-01-21T22:25:07.093975+0000","last_undegraded":"2024-01-21T22:25:07.093975+0000","last_fullsized":"2024-01-21T22:25:07.093975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:12:11.958115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.152598+0000","last_change":"2024-01-21T22:25:03.720257+0000","last_active":"2024-01-21T22:25:07.152598+0000","last_peered":"2024-01-21T22:25:07.152598+0000","last_clean":"2024-01-21T22:25:07.152598+0000","last_became_active":"2024-01-21T22:25:03.720009+0000","last_became_peered":"2024-01-21T22:25:03.720009+0000","last_unstale":"2024-01-21T22:25:07.152598+0000","last_undegraded":"2024-01-21T22:25:07.152598+0000","last_fullsized":"2024-01-21T22:25:07.152598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:37:56.569677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.144048+0000","last_change":"2024-01-21T22:25:04.883112+0000","last_active":"2024-01-21T22:25:07.144048+0000","last_peered":"2024-01-21T22:25:07.144048+0000","last_clean":"2024-01-21T22:25:07.144048+0000","last_became_active":"2024-01-21T22:25:04.882767+0000","last_became_peered":"2024-01-21T22:25:04.882767+0000","last_unstale":"2024-01-21T22:25:07.144048+0000","last_undegraded":"2024-01-21T22:25:07.144048+0000","last_fullsized":"2024-01-21T22:25:07.144048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:18:03.439738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.303803+0000","last_change":"2024-01-21T22:25:03.703516+0000","last_active":"2024-01-21T22:25:07.303803+0000","last_peered":"2024-01-21T22:25:07.303803+0000","last_clean":"2024-01-21T22:25:07.303803+0000","last_became_active":"2024-01-21T22:25:03.703256+0000","last_became_peered":"2024-01-21T22:25:03.703256+0000","last_unstale":"2024-01-21T22:25:07.303803+0000","last_undegraded":"2024-01-21T22:25:07.303803+0000","last_fullsized":"2024-01-21T22:25:07.303803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:57:42.881371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286834+0000","last_change":"2024-01-21T22:25:04.884129+0000","last_active":"2024-01-21T22:25:07.286834+0000","last_peered":"2024-01-21T22:25:07.286834+0000","last_clean":"2024-01-21T22:25:07.286834+0000","last_became_active":"2024-01-21T22:25:04.883839+0000","last_became_peered":"2024-01-21T22:25:04.883839+0000","last_unstale":"2024-01-21T22:25:07.286834+0000","last_undegraded":"2024-01-21T22:25:07.286834+0000","last_fullsized":"2024-01-21T22:25:07.286834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:43:55.049706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298049+0000","last_change":"2024-01-21T22:25:03.708366+0000","last_active":"2024-01-21T22:25:07.298049+0000","last_peered":"2024-01-21T22:25:07.298049+0000","last_clean":"2024-01-21T22:25:07.298049+0000","last_became_active":"2024-01-21T22:25:03.708101+0000","last_became_peered":"2024-01-21T22:25:03.708101+0000","last_unstale":"2024-01-21T22:25:07.298049+0000","last_undegraded":"2024-01-21T22:25:07.298049+0000","last_fullsized":"2024-01-21T22:25:07.298049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:57:42.881371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286513+0000","last_change":"2024-01-21T22:25:04.877993+0000","last_active":"2024-01-21T22:25:07.286513+0000","last_peered":"2024-01-21T22:25:07.286513+0000","last_clean":"2024-01-21T22:25:07.286513+0000","last_became_active":"2024-01-21T22:25:04.877661+0000","last_became_peered":"2024-01-21T22:25:04.877661+0000","last_unstale":"2024-01-21T22:25:07.286513+0000","last_undegraded":"2024-01-21T22:25:07.286513+0000","last_fullsized":"2024-01-21T22:25:07.286513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:20:40.947205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.878348+0000","last_change":"2024-01-21T22:25:03.721559+0000","last_active":"2024-01-21T22:25:05.878348+0000","last_peered":"2024-01-21T22:25:05.878348+0000","last_clean":"2024-01-21T22:25:05.878348+0000","last_became_active":"2024-01-21T22:25:03.721281+0000","last_became_peered":"2024-01-21T22:25:03.721281+0000","last_unstale":"2024-01-21T22:25:05.878348+0000","last_undegraded":"2024-01-21T22:25:05.878348+0000","last_fullsized":"2024-01-21T22:25:05.878348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:55:19.679388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.094279+0000","last_change":"2024-01-21T22:25:04.883087+0000","last_active":"2024-01-21T22:25:07.094279+0000","last_peered":"2024-01-21T22:25:07.094279+0000","last_clean":"2024-01-21T22:25:07.094279+0000","last_became_active":"2024-01-21T22:25:04.880227+0000","last_became_peered":"2024-01-21T22:25:04.880227+0000","last_unstale":"2024-01-21T22:25:07.094279+0000","last_undegraded":"2024-01-21T22:25:07.094279+0000","last_fullsized":"2024-01-21T22:25:07.094279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:58:09.653538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.880999+0000","last_change":"2024-01-21T22:24:52.968324+0000","last_active":"2024-01-21T22:25:05.880999+0000","last_peered":"2024-01-21T22:25:05.880999+0000","last_clean":"2024-01-21T22:25:05.880999+0000","last_became_active":"2024-01-21T22:24:52.967018+0000","last_became_peered":"2024-01-21T22:24:52.967018+0000","last_unstale":"2024-01-21T22:25:05.880999+0000","last_undegraded":"2024-01-21T22:25:05.880999+0000","last_fullsized":"2024-01-21T22:25:05.880999+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:24:50.312540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:24:50.312540+0000","last_clean_scrub_stamp":"2024-01-21T22:24:50.312540+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:52:04.800073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.522445+0000","last_change":"2024-01-21T22:25:03.707407+0000","last_active":"2024-01-21T22:25:05.522445+0000","last_peered":"2024-01-21T22:25:05.522445+0000","last_clean":"2024-01-21T22:25:05.522445+0000","last_became_active":"2024-01-21T22:25:03.707085+0000","last_became_peered":"2024-01-21T22:25:03.707085+0000","last_unstale":"2024-01-21T22:25:05.522445+0000","last_undegraded":"2024-01-21T22:25:05.522445+0000","last_fullsized":"2024-01-21T22:25:05.522445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:44:22.501512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151755+0000","last_change":"2024-01-21T22:25:04.868564+0000","last_active":"2024-01-21T22:25:07.151755+0000","last_peered":"2024-01-21T22:25:07.151755+0000","last_clean":"2024-01-21T22:25:07.151755+0000","last_became_active":"2024-01-21T22:25:04.868247+0000","last_became_peered":"2024-01-21T22:25:04.868247+0000","last_unstale":"2024-01-21T22:25:07.151755+0000","last_undegraded":"2024-01-21T22:25:07.151755+0000","last_fullsized":"2024-01-21T22:25:07.151755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:30:50.039501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.521811+0000","last_change":"2024-01-21T22:25:03.706070+0000","last_active":"2024-01-21T22:25:05.521811+0000","last_peered":"2024-01-21T22:25:05.521811+0000","last_clean":"2024-01-21T22:25:05.521811+0000","last_became_active":"2024-01-21T22:25:03.705785+0000","last_became_peered":"2024-01-21T22:25:03.705785+0000","last_unstale":"2024-01-21T22:25:05.521811+0000","last_undegraded":"2024-01-21T22:25:05.521811+0000","last_fullsized":"2024-01-21T22:25:05.521811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:30.929444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304132+0000","last_change":"2024-01-21T22:25:04.845436+0000","last_active":"2024-01-21T22:25:07.304132+0000","last_peered":"2024-01-21T22:25:07.304132+0000","last_clean":"2024-01-21T22:25:07.304132+0000","last_became_active":"2024-01-21T22:25:04.845274+0000","last_became_peered":"2024-01-21T22:25:04.845274+0000","last_unstale":"2024-01-21T22:25:07.304132+0000","last_undegraded":"2024-01-21T22:25:07.304132+0000","last_fullsized":"2024-01-21T22:25:07.304132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:08:36.686787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.878258+0000","last_change":"2024-01-21T22:25:03.709174+0000","last_active":"2024-01-21T22:25:05.878258+0000","last_peered":"2024-01-21T22:25:05.878258+0000","last_clean":"2024-01-21T22:25:05.878258+0000","last_became_active":"2024-01-21T22:25:03.708894+0000","last_became_peered":"2024-01-21T22:25:03.708894+0000","last_unstale":"2024-01-21T22:25:05.878258+0000","last_undegraded":"2024-01-21T22:25:05.878258+0000","last_fullsized":"2024-01-21T22:25:05.878258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:19:26.591140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297083+0000","last_change":"2024-01-21T22:25:04.711947+0000","last_active":"2024-01-21T22:25:07.297083+0000","last_peered":"2024-01-21T22:25:07.297083+0000","last_clean":"2024-01-21T22:25:07.297083+0000","last_became_active":"2024-01-21T22:25:04.711694+0000","last_became_peered":"2024-01-21T22:25:04.711694+0000","last_unstale":"2024-01-21T22:25:07.297083+0000","last_undegraded":"2024-01-21T22:25:07.297083+0000","last_fullsized":"2024-01-21T22:25:07.297083+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:37:57.817795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.522504+0000","last_change":"2024-01-21T22:25:03.706355+0000","last_active":"2024-01-21T22:25:05.522504+0000","last_peered":"2024-01-21T22:25:05.522504+0000","last_clean":"2024-01-21T22:25:05.522504+0000","last_became_active":"2024-01-21T22:25:03.703876+0000","last_became_peered":"2024-01-21T22:25:03.703876+0000","last_unstale":"2024-01-21T22:25:05.522504+0000","last_undegraded":"2024-01-21T22:25:05.522504+0000","last_fullsized":"2024-01-21T22:25:05.522504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:42:38.729006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.285835+0000","last_change":"2024-01-21T22:25:04.885997+0000","last_active":"2024-01-21T22:25:07.285835+0000","last_peered":"2024-01-21T22:25:07.285835+0000","last_clean":"2024-01-21T22:25:07.285835+0000","last_became_active":"2024-01-21T22:25:04.884162+0000","last_became_peered":"2024-01-21T22:25:04.884162+0000","last_unstale":"2024-01-21T22:25:07.285835+0000","last_undegraded":"2024-01-21T22:25:07.285835+0000","last_fullsized":"2024-01-21T22:25:07.285835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:44:55.158990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.521383+0000","last_change":"2024-01-21T22:25:03.708300+0000","last_active":"2024-01-21T22:25:05.521383+0000","last_peered":"2024-01-21T22:25:05.521383+0000","last_clean":"2024-01-21T22:25:05.521383+0000","last_became_active":"2024-01-21T22:25:03.708036+0000","last_became_peered":"2024-01-21T22:25:03.708036+0000","last_unstale":"2024-01-21T22:25:05.521383+0000","last_undegraded":"2024-01-21T22:25:05.521383+0000","last_fullsized":"2024-01-21T22:25:05.521383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:23:51.196159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.143828+0000","last_change":"2024-01-21T22:25:04.844713+0000","last_active":"2024-01-21T22:25:07.143828+0000","last_peered":"2024-01-21T22:25:07.143828+0000","last_clean":"2024-01-21T22:25:07.143828+0000","last_became_active":"2024-01-21T22:25:04.844008+0000","last_became_peered":"2024-01-21T22:25:04.844008+0000","last_unstale":"2024-01-21T22:25:07.143828+0000","last_undegraded":"2024-01-21T22:25:07.143828+0000","last_fullsized":"2024-01-21T22:25:07.143828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:33:25.760064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.523113+0000","last_change":"2024-01-21T22:25:03.699093+0000","last_active":"2024-01-21T22:25:05.523113+0000","last_peered":"2024-01-21T22:25:05.523113+0000","last_clean":"2024-01-21T22:25:05.523113+0000","last_became_active":"2024-01-21T22:25:03.698443+0000","last_became_peered":"2024-01-21T22:25:03.698443+0000","last_unstale":"2024-01-21T22:25:05.523113+0000","last_undegraded":"2024-01-21T22:25:05.523113+0000","last_fullsized":"2024-01-21T22:25:05.523113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:20:39.699087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304411+0000","last_change":"2024-01-21T22:25:04.842733+0000","last_active":"2024-01-21T22:25:07.304411+0000","last_peered":"2024-01-21T22:25:07.304411+0000","last_clean":"2024-01-21T22:25:07.304411+0000","last_became_active":"2024-01-21T22:25:04.842479+0000","last_became_peered":"2024-01-21T22:25:04.842479+0000","last_unstale":"2024-01-21T22:25:07.304411+0000","last_undegraded":"2024-01-21T22:25:07.304411+0000","last_fullsized":"2024-01-21T22:25:07.304411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:51:06.931790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151980+0000","last_change":"2024-01-21T22:25:03.720306+0000","last_active":"2024-01-21T22:25:07.151980+0000","last_peered":"2024-01-21T22:25:07.151980+0000","last_clean":"2024-01-21T22:25:07.151980+0000","last_became_active":"2024-01-21T22:25:03.720067+0000","last_became_peered":"2024-01-21T22:25:03.720067+0000","last_unstale":"2024-01-21T22:25:07.151980+0000","last_undegraded":"2024-01-21T22:25:07.151980+0000","last_fullsized":"2024-01-21T22:25:07.151980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:52:22.985590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151977+0000","last_change":"2024-01-21T22:25:04.859628+0000","last_active":"2024-01-21T22:25:07.151977+0000","last_peered":"2024-01-21T22:25:07.151977+0000","last_clean":"2024-01-21T22:25:07.151977+0000","last_became_active":"2024-01-21T22:25:04.859270+0000","last_became_peered":"2024-01-21T22:25:04.859270+0000","last_unstale":"2024-01-21T22:25:07.151977+0000","last_undegraded":"2024-01-21T22:25:07.151977+0000","last_fullsized":"2024-01-21T22:25:07.151977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:56:24.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.282102+0000","last_change":"2024-01-21T22:25:03.722006+0000","last_active":"2024-01-21T22:25:07.282102+0000","last_peered":"2024-01-21T22:25:07.282102+0000","last_clean":"2024-01-21T22:25:07.282102+0000","last_became_active":"2024-01-21T22:25:03.721275+0000","last_became_peered":"2024-01-21T22:25:03.721275+0000","last_unstale":"2024-01-21T22:25:07.282102+0000","last_undegraded":"2024-01-21T22:25:07.282102+0000","last_fullsized":"2024-01-21T22:25:07.282102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:43:53.801587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151916+0000","last_change":"2024-01-21T22:25:04.858864+0000","last_active":"2024-01-21T22:25:07.151916+0000","last_peered":"2024-01-21T22:25:07.151916+0000","last_clean":"2024-01-21T22:25:07.151916+0000","last_became_active":"2024-01-21T22:25:04.858586+0000","last_became_peered":"2024-01-21T22:25:04.858586+0000","last_unstale":"2024-01-21T22:25:07.151916+0000","last_undegraded":"2024-01-21T22:25:07.151916+0000","last_fullsized":"2024-01-21T22:25:07.151916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:32.177562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.609941+0000","last_change":"2024-01-21T22:25:03.718436+0000","last_active":"2024-01-21T22:25:05.609941+0000","last_peered":"2024-01-21T22:25:05.609941+0000","last_clean":"2024-01-21T22:25:05.609941+0000","last_became_active":"2024-01-21T22:25:03.718171+0000","last_became_peered":"2024-01-21T22:25:03.718171+0000","last_unstale":"2024-01-21T22:25:05.609941+0000","last_undegraded":"2024-01-21T22:25:05.609941+0000","last_fullsized":"2024-01-21T22:25:05.609941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:05:40.702113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093711+0000","last_change":"2024-01-21T22:25:04.875736+0000","last_active":"2024-01-21T22:25:07.093711+0000","last_peered":"2024-01-21T22:25:07.093711+0000","last_clean":"2024-01-21T22:25:07.093711+0000","last_became_active":"2024-01-21T22:25:04.875368+0000","last_became_peered":"2024-01-21T22:25:04.875368+0000","last_unstale":"2024-01-21T22:25:07.093711+0000","last_undegraded":"2024-01-21T22:25:07.093711+0000","last_fullsized":"2024-01-21T22:25:07.093711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:39:31.273250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.877906+0000","last_change":"2024-01-21T22:25:03.722820+0000","last_active":"2024-01-21T22:25:05.877906+0000","last_peered":"2024-01-21T22:25:05.877906+0000","last_clean":"2024-01-21T22:25:05.877906+0000","last_became_active":"2024-01-21T22:25:03.722525+0000","last_became_peered":"2024-01-21T22:25:03.722525+0000","last_unstale":"2024-01-21T22:25:05.877906+0000","last_undegraded":"2024-01-21T22:25:05.877906+0000","last_fullsized":"2024-01-21T22:25:05.877906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:40:04.306585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151604+0000","last_change":"2024-01-21T22:25:04.857451+0000","last_active":"2024-01-21T22:25:07.151604+0000","last_peered":"2024-01-21T22:25:07.151604+0000","last_clean":"2024-01-21T22:25:07.151604+0000","last_became_active":"2024-01-21T22:25:04.857205+0000","last_became_peered":"2024-01-21T22:25:04.857205+0000","last_unstale":"2024-01-21T22:25:07.151604+0000","last_undegraded":"2024-01-21T22:25:07.151604+0000","last_fullsized":"2024-01-21T22:25:07.151604+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:12.153042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.877746+0000","last_change":"2024-01-21T22:25:03.708574+0000","last_active":"2024-01-21T22:25:05.877746+0000","last_peered":"2024-01-21T22:25:05.877746+0000","last_clean":"2024-01-21T22:25:05.877746+0000","last_became_active":"2024-01-21T22:25:03.708318+0000","last_became_peered":"2024-01-21T22:25:03.708318+0000","last_unstale":"2024-01-21T22:25:05.877746+0000","last_undegraded":"2024-01-21T22:25:05.877746+0000","last_fullsized":"2024-01-21T22:25:05.877746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:50:40.058463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.296286+0000","last_change":"2024-01-21T22:25:04.843079+0000","last_active":"2024-01-21T22:25:07.296286+0000","last_peered":"2024-01-21T22:25:07.296286+0000","last_clean":"2024-01-21T22:25:07.296286+0000","last_became_active":"2024-01-21T22:25:04.842815+0000","last_became_peered":"2024-01-21T22:25:04.842815+0000","last_unstale":"2024-01-21T22:25:07.296286+0000","last_undegraded":"2024-01-21T22:25:07.296286+0000","last_fullsized":"2024-01-21T22:25:07.296286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:16:41.440927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.281616+0000","last_change":"2024-01-21T22:25:03.710506+0000","last_active":"2024-01-21T22:25:07.281616+0000","last_peered":"2024-01-21T22:25:07.281616+0000","last_clean":"2024-01-21T22:25:07.281616+0000","last_became_active":"2024-01-21T22:25:03.710262+0000","last_became_peered":"2024-01-21T22:25:03.710262+0000","last_unstale":"2024-01-21T22:25:07.281616+0000","last_undegraded":"2024-01-21T22:25:07.281616+0000","last_fullsized":"2024-01-21T22:25:07.281616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:25:49.132306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093352+0000","last_change":"2024-01-21T22:25:04.876469+0000","last_active":"2024-01-21T22:25:07.093352+0000","last_peered":"2024-01-21T22:25:07.093352+0000","last_clean":"2024-01-21T22:25:07.093352+0000","last_became_active":"2024-01-21T22:25:04.875619+0000","last_became_peered":"2024-01-21T22:25:04.875619+0000","last_unstale":"2024-01-21T22:25:07.093352+0000","last_undegraded":"2024-01-21T22:25:07.093352+0000","last_fullsized":"2024-01-21T22:25:07.093352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:30:20.167607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.094159+0000","last_change":"2024-01-21T22:25:04.891067+0000","last_active":"2024-01-21T22:25:07.094159+0000","last_peered":"2024-01-21T22:25:07.094159+0000","last_clean":"2024-01-21T22:25:07.094159+0000","last_became_active":"2024-01-21T22:25:04.890515+0000","last_became_peered":"2024-01-21T22:25:04.890515+0000","last_unstale":"2024-01-21T22:25:07.094159+0000","last_undegraded":"2024-01-21T22:25:07.094159+0000","last_fullsized":"2024-01-21T22:25:07.094159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:25:48.726591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.158438+0000","last_change":"2024-01-21T22:25:03.719412+0000","last_active":"2024-01-21T22:25:07.158438+0000","last_peered":"2024-01-21T22:25:07.158438+0000","last_clean":"2024-01-21T22:25:07.158438+0000","last_became_active":"2024-01-21T22:25:03.718163+0000","last_became_peered":"2024-01-21T22:25:03.718163+0000","last_unstale":"2024-01-21T22:25:07.158438+0000","last_undegraded":"2024-01-21T22:25:07.158438+0000","last_fullsized":"2024-01-21T22:25:07.158438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:56:22.817420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304192+0000","last_change":"2024-01-21T22:25:04.889166+0000","last_active":"2024-01-21T22:25:07.304192+0000","last_peered":"2024-01-21T22:25:07.304192+0000","last_clean":"2024-01-21T22:25:07.304192+0000","last_became_active":"2024-01-21T22:25:04.888910+0000","last_became_peered":"2024-01-21T22:25:04.888910+0000","last_unstale":"2024-01-21T22:25:07.304192+0000","last_undegraded":"2024-01-21T22:25:07.304192+0000","last_fullsized":"2024-01-21T22:25:07.304192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:55:53.892439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.156719+0000","last_change":"2024-01-21T22:25:03.718932+0000","last_active":"2024-01-21T22:25:07.156719+0000","last_peered":"2024-01-21T22:25:07.156719+0000","last_clean":"2024-01-21T22:25:07.156719+0000","last_became_active":"2024-01-21T22:25:03.718649+0000","last_became_peered":"2024-01-21T22:25:03.718649+0000","last_unstale":"2024-01-21T22:25:07.156719+0000","last_undegraded":"2024-01-21T22:25:07.156719+0000","last_fullsized":"2024-01-21T22:25:07.156719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:30.929444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297922+0000","last_change":"2024-01-21T22:25:04.715429+0000","last_active":"2024-01-21T22:25:07.297922+0000","last_peered":"2024-01-21T22:25:07.297922+0000","last_clean":"2024-01-21T22:25:07.297922+0000","last_became_active":"2024-01-21T22:25:04.714720+0000","last_became_peered":"2024-01-21T22:25:04.714720+0000","last_unstale":"2024-01-21T22:25:07.297922+0000","last_undegraded":"2024-01-21T22:25:07.297922+0000","last_fullsized":"2024-01-21T22:25:07.297922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:44:23.749631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298298+0000","last_change":"2024-01-21T22:25:03.703289+0000","last_active":"2024-01-21T22:25:07.298298+0000","last_peered":"2024-01-21T22:25:07.298298+0000","last_clean":"2024-01-21T22:25:07.298298+0000","last_became_active":"2024-01-21T22:25:03.703020+0000","last_became_peered":"2024-01-21T22:25:03.703020+0000","last_unstale":"2024-01-21T22:25:07.298298+0000","last_undegraded":"2024-01-21T22:25:07.298298+0000","last_fullsized":"2024-01-21T22:25:07.298298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:10.904923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298304+0000","last_change":"2024-01-21T22:25:04.715894+0000","last_active":"2024-01-21T22:25:07.298304+0000","last_peered":"2024-01-21T22:25:07.298304+0000","last_clean":"2024-01-21T22:25:07.298304+0000","last_became_active":"2024-01-21T22:25:04.715652+0000","last_became_peered":"2024-01-21T22:25:04.715652+0000","last_unstale":"2024-01-21T22:25:07.298304+0000","last_undegraded":"2024-01-21T22:25:07.298304+0000","last_fullsized":"2024-01-21T22:25:07.298304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:13:04.601911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"23'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.303733+0000","last_change":"2024-01-21T22:25:03.702339+0000","last_active":"2024-01-21T22:25:07.303733+0000","last_peered":"2024-01-21T22:25:07.303733+0000","last_clean":"2024-01-21T22:25:07.303733+0000","last_became_active":"2024-01-21T22:25:03.702087+0000","last_became_peered":"2024-01-21T22:25:03.702087+0000","last_unstale":"2024-01-21T22:25:07.303733+0000","last_undegraded":"2024-01-21T22:25:07.303733+0000","last_fullsized":"2024-01-21T22:25:07.303733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:51:05.683671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542156,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":245760,"data_stored":85152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]}]},{"osd":6,"up_from":14,"seq":60129542156,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":85028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542156,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":839680,"data_stored":675488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542156,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":87312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]}]},{"osd":3,"up_from":14,"seq":60129542157,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":85610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5489999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3660000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.488}]}]},{"osd":2,"up_from":14,"seq":60129542155,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":249856,"data_stored":86640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542156,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":835584,"data_stored":675466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.28}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4610000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542157,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":245760,"data_stored":85084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.458}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2769999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-21T22:25:25.528 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-21T22:25:25.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.767+0000 7f0795800640 1 Processor -- start 2024-01-21T22:25:25.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.767+0000 7f0795800640 1 -- start start 2024-01-21T22:25:25.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.768+0000 7f0795800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f0790151e10 0x7f079015ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:25.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.768+0000 7f0795800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f079015b3a0 0x7f07900c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:25.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.769+0000 7f0795800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07901583d0 con 0x7f0790151a30 2024-01-21T22:25:25.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.769+0000 7f0795800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f079006c7b0 con 0x7f0790151e10 2024-01-21T22:25:25.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.769+0000 7f0795800640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0790157890 con 0x7f079015b3a0 2024-01-21T22:25:25.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.769+0000 7f078ea00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0790151a30 0x7f07900c5ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:38056/0 (socket says 172.21.15.138:38056) 2024-01-21T22:25:25.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.769+0000 7f078ea00640 1 -- 172.21.15.138:0/613527158 learned_addr learned my addr 172.21.15.138:0/613527158 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:25.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.769+0000 7f078f400640 1 --2- 172.21.15.138:0/613527158 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f079015b3a0 0x7f07900c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:25.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.770+0000 7f0785e00640 1 --2- 172.21.15.138:0/613527158 >> v2:172.21.15.152:3300/0 conn(0x7f0790151e10 0x7f079015ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:25.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.772+0000 7f0785e00640 1 -- 172.21.15.138:0/613527158 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f079015b3a0 msgr2=0x7f07900c47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:25.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.772+0000 7f0785e00640 1 --2- 172.21.15.138:0/613527158 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f079015b3a0 0x7f07900c47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:25.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.772+0000 7f0785e00640 1 -- 172.21.15.138:0/613527158 >> v1:172.21.15.138:6789/0 conn(0x7f0790151a30 legacy=0x7f07900c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:25.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.772+0000 7f0785e00640 1 -- 172.21.15.138:0/613527158 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079006c4d0 con 0x7f0790151e10 2024-01-21T22:25:25.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.774+0000 7f0785e00640 1 --2- 172.21.15.138:0/613527158 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0790151e10 0x7f079015ae60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0778009d90 tx=0x7f077802bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c156d0f3501ea72 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:25.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.779+0000 7f078e000640 1 -- 172.21.15.138:0/613527158 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077802cce0 con 0x7f0790151e10 2024-01-21T22:25:25.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.779+0000 7f078e000640 1 -- 172.21.15.138:0/613527158 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077802ce40 con 0x7f0790151e10 2024-01-21T22:25:25.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.779+0000 7f078e000640 1 -- 172.21.15.138:0/613527158 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0778034660 con 0x7f0790151e10 2024-01-21T22:25:25.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.779+0000 7f0795800640 1 -- 172.21.15.138:0/613527158 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0790151e10 msgr2=0x7f079015ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:25.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.779+0000 7f0795800640 1 --2- 172.21.15.138:0/613527158 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0790151e10 0x7f079015ae60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0778009d90 tx=0x7f077802bf30 comp rx=0 tx=0).stop 2024-01-21T22:25:25.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.779+0000 7f0795800640 1 -- 172.21.15.138:0/613527158 shutdown_connections 2024-01-21T22:25:25.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.779+0000 7f0795800640 1 --2- 172.21.15.138:0/613527158 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f079015b3a0 0x7f07900c47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:25.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.779+0000 7f0795800640 1 --2- 172.21.15.138:0/613527158 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0790151e10 0x7f079015ae60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:25.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.779+0000 7f0795800640 1 -- 172.21.15.138:0/613527158 >> 172.21.15.138:0/613527158 conn(0x7f07900cb0b0 msgr2=0x7f07900cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:25.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.779+0000 7f0795800640 1 -- 172.21.15.138:0/613527158 shutdown_connections 2024-01-21T22:25:25.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.779+0000 7f0795800640 1 -- 172.21.15.138:0/613527158 wait complete. 2024-01-21T22:25:25.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.781+0000 7f0795800640 1 Processor -- start 2024-01-21T22:25:25.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.781+0000 7f0795800640 1 -- start start 2024-01-21T22:25:25.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.783+0000 7f0795800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0790151a30 0x7f0790159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:25.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.783+0000 7f0795800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f079015b3a0 0x7f079014ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:25.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.783+0000 7f0795800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0790155020 0x7f079014ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:25.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.783+0000 7f0795800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07901e3e40 con 0x7f0790155020 2024-01-21T22:25:25.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.783+0000 7f0795800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07901e2f50 con 0x7f0790151a30 2024-01-21T22:25:25.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.783+0000 7f0795800640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07901e3230 con 0x7f079015b3a0 2024-01-21T22:25:25.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.783+0000 7f0785e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f079015b3a0 0x7f079014ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:25.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.783+0000 7f078f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0790155020 0x7f079014ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:25.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.784+0000 7f078ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0790151a30 0x7f0790159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:25.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.784+0000 7f0785e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f079015b3a0 0x7f079014ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:42748/0 (socket says 172.21.15.138:42748) 2024-01-21T22:25:25.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.784+0000 7f078f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0790155020 0x7f079014ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53278/0 (socket says 172.21.15.138:53278) 2024-01-21T22:25:25.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.784+0000 7f078ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0790151a30 0x7f0790159f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:36598/0 (socket says 172.21.15.138:36598) 2024-01-21T22:25:25.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.784+0000 7f0785e00640 1 -- 172.21.15.138:0/253331028 learned_addr learned my addr 172.21.15.138:0/253331028 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:25.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.787+0000 7f0785e00640 1 -- 172.21.15.138:0/253331028 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0790151a30 msgr2=0x7f0790159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:25.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.787+0000 7f0785e00640 1 --2- 172.21.15.138:0/253331028 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0790151a30 0x7f0790159f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:25.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.787+0000 7f0785e00640 1 -- 172.21.15.138:0/253331028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0790155020 msgr2=0x7f079014ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:25.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.787+0000 7f0785e00640 1 --2- 172.21.15.138:0/253331028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0790155020 0x7f079014ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:25.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.787+0000 7f0785e00640 1 -- 172.21.15.138:0/253331028 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0778009a70 con 0x7f079015b3a0 2024-01-21T22:25:25.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.789+0000 7f0785e00640 1 --2- 172.21.15.138:0/253331028 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f079015b3a0 0x7f079014ea20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0778009d90 tx=0x7f0778004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:25.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.794+0000 7f0787400640 1 -- 172.21.15.138:0/253331028 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0778009450 con 0x7f079015b3a0 2024-01-21T22:25:25.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.794+0000 7f0787400640 1 -- 172.21.15.138:0/253331028 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077803d400 con 0x7f079015b3a0 2024-01-21T22:25:25.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.794+0000 7f0795800640 1 -- 172.21.15.138:0/253331028 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07901e3510 con 0x7f079015b3a0 2024-01-21T22:25:25.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.794+0000 7f0787400640 1 -- 172.21.15.138:0/253331028 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077803d790 con 0x7f079015b3a0 2024-01-21T22:25:25.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.794+0000 7f0795800640 1 -- 172.21.15.138:0/253331028 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07901e3ad0 con 0x7f079015b3a0 2024-01-21T22:25:25.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.795+0000 7f0795800640 1 -- 172.21.15.138:0/253331028 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07901e3c90 con 0x7f079015b3a0 2024-01-21T22:25:25.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.816+0000 7f0787400640 1 -- 172.21.15.138:0/253331028 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f077803b040 con 0x7f079015b3a0 2024-01-21T22:25:25.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.817+0000 7f0787400640 1 --2- 172.21.15.138:0/253331028 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f076806b380 0x7f076806d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:25.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.818+0000 7f078ea00640 1 --2- 172.21.15.138:0/253331028 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f076806b380 0x7f076806d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:25.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:25.818+0000 7f078ea00640 1 --2- 172.21.15.138:0/253331028 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f076806b380 0x7f076806d840 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0780007fd0 tx=0x7f0780007e80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:26.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.043+0000 7f0787400640 1 -- 172.21.15.138:0/253331028 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f077807ab50 con 0x7f079015b3a0 2024-01-21T22:25:26.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.053+0000 7f0787400640 1 -- 172.21.15.138:0/253331028 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f0778076e90 con 0x7f079015b3a0 2024-01-21T22:25:26.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.269+0000 7f0795800640 1 -- 172.21.15.138:0/253331028 --> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0790158860 con 0x7f076806b380 2024-01-21T22:25:26.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.276+0000 7f0787400640 1 -- 172.21.15.138:0/253331028 <== mgr.4111 v2:172.21.15.152:6800/314625564 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264479 (secure 0 0 0) 0x7f0780001460 con 0x7f076806b380 2024-01-21T22:25:26.282 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:25:26.283 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-01-21T22:25:26.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.282+0000 7f0795800640 1 -- 172.21.15.138:0/253331028 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f076806b380 msgr2=0x7f076806d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:26.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.282+0000 7f0795800640 1 --2- 172.21.15.138:0/253331028 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f076806b380 0x7f076806d840 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0780007fd0 tx=0x7f0780007e80 comp rx=0 tx=0).stop 2024-01-21T22:25:26.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.282+0000 7f0795800640 1 -- 172.21.15.138:0/253331028 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f079015b3a0 msgr2=0x7f079014ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:26.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.282+0000 7f0795800640 1 --2- 172.21.15.138:0/253331028 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f079015b3a0 0x7f079014ea20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0778009d90 tx=0x7f0778004a40 comp rx=0 tx=0).stop 2024-01-21T22:25:26.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.282+0000 7f0795800640 1 -- 172.21.15.138:0/253331028 shutdown_connections 2024-01-21T22:25:26.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.282+0000 7f0795800640 1 --2- 172.21.15.138:0/253331028 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f076806b380 0x7f076806d840 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:26.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.282+0000 7f0795800640 1 --2- 172.21.15.138:0/253331028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0790155020 0x7f079014ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:26.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.282+0000 7f0795800640 1 --2- 172.21.15.138:0/253331028 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f079015b3a0 0x7f079014ea20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:26.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.282+0000 7f0795800640 1 --2- 172.21.15.138:0/253331028 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0790151a30 0x7f0790159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:26.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.282+0000 7f0795800640 1 -- 172.21.15.138:0/253331028 >> 172.21.15.138:0/253331028 conn(0x7f07900cb0b0 msgr2=0x7f0790150ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:26.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.282+0000 7f0795800640 1 -- 172.21.15.138:0/253331028 shutdown_connections 2024-01-21T22:25:26.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.282+0000 7f0795800640 1 -- 172.21.15.138:0/253331028 wait complete. 2024-01-21T22:25:26.301 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-01-21T22:25:25.219307+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219240,"kb_used_data":3080,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755480,"statfs":{"total":773094113280,"available":772869611520,"internally_reserved":0,"allocated":3153920,"data_stored":1865780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002962"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286921+0000","last_change":"2024-01-21T22:25:04.888366+0000","last_active":"2024-01-21T22:25:07.286921+0000","last_peered":"2024-01-21T22:25:07.286921+0000","last_clean":"2024-01-21T22:25:07.286921+0000","last_became_active":"2024-01-21T22:25:04.888060+0000","last_became_peered":"2024-01-21T22:25:04.888060+0000","last_unstale":"2024-01-21T22:25:07.286921+0000","last_undegraded":"2024-01-21T22:25:07.286921+0000","last_fullsized":"2024-01-21T22:25:07.286921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:32:13.512213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298177+0000","last_change":"2024-01-21T22:25:04.843704+0000","last_active":"2024-01-21T22:25:07.298177+0000","last_peered":"2024-01-21T22:25:07.298177+0000","last_clean":"2024-01-21T22:25:07.298177+0000","last_became_active":"2024-01-21T22:25:04.843471+0000","last_became_peered":"2024-01-21T22:25:04.843471+0000","last_unstale":"2024-01-21T22:25:07.298177+0000","last_undegraded":"2024-01-21T22:25:07.298177+0000","last_fullsized":"2024-01-21T22:25:07.298177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:34:24.822768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298055+0000","last_change":"2024-01-21T22:25:04.858021+0000","last_active":"2024-01-21T22:25:07.298055+0000","last_peered":"2024-01-21T22:25:07.298055+0000","last_clean":"2024-01-21T22:25:07.298055+0000","last_became_active":"2024-01-21T22:25:04.857685+0000","last_became_peered":"2024-01-21T22:25:04.857685+0000","last_unstale":"2024-01-21T22:25:07.298055+0000","last_undegraded":"2024-01-21T22:25:07.298055+0000","last_fullsized":"2024-01-21T22:25:07.298055+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:29:02.950903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.144191+0000","last_change":"2024-01-21T22:25:04.858941+0000","last_active":"2024-01-21T22:25:07.144191+0000","last_peered":"2024-01-21T22:25:07.144191+0000","last_clean":"2024-01-21T22:25:07.144191+0000","last_became_active":"2024-01-21T22:25:04.858278+0000","last_became_peered":"2024-01-21T22:25:04.858278+0000","last_unstale":"2024-01-21T22:25:07.144191+0000","last_undegraded":"2024-01-21T22:25:07.144191+0000","last_fullsized":"2024-01-21T22:25:07.144191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T10:03:50.093635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286258+0000","last_change":"2024-01-21T22:25:04.881014+0000","last_active":"2024-01-21T22:25:07.286258+0000","last_peered":"2024-01-21T22:25:07.286258+0000","last_clean":"2024-01-21T22:25:07.286258+0000","last_became_active":"2024-01-21T22:25:04.878457+0000","last_became_peered":"2024-01-21T22:25:04.878457+0000","last_unstale":"2024-01-21T22:25:07.286258+0000","last_undegraded":"2024-01-21T22:25:07.286258+0000","last_fullsized":"2024-01-21T22:25:07.286258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:34:24.822768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.143791+0000","last_change":"2024-01-21T22:25:04.859776+0000","last_active":"2024-01-21T22:25:07.143791+0000","last_peered":"2024-01-21T22:25:07.143791+0000","last_clean":"2024-01-21T22:25:07.143791+0000","last_became_active":"2024-01-21T22:25:04.858731+0000","last_became_peered":"2024-01-21T22:25:04.858731+0000","last_unstale":"2024-01-21T22:25:07.143791+0000","last_undegraded":"2024-01-21T22:25:07.143791+0000","last_fullsized":"2024-01-21T22:25:07.143791+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:01:30.091588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.285525+0000","last_change":"2024-01-21T22:25:04.881392+0000","last_active":"2024-01-21T22:25:07.285525+0000","last_peered":"2024-01-21T22:25:07.285525+0000","last_clean":"2024-01-21T22:25:07.285525+0000","last_became_active":"2024-01-21T22:25:04.881090+0000","last_became_peered":"2024-01-21T22:25:04.881090+0000","last_unstale":"2024-01-21T22:25:07.285525+0000","last_undegraded":"2024-01-21T22:25:07.285525+0000","last_fullsized":"2024-01-21T22:25:07.285525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:25:50.380424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297019+0000","last_change":"2024-01-21T22:25:04.879186+0000","last_active":"2024-01-21T22:25:07.297019+0000","last_peered":"2024-01-21T22:25:07.297019+0000","last_clean":"2024-01-21T22:25:07.297019+0000","last_became_active":"2024-01-21T22:25:04.878824+0000","last_became_peered":"2024-01-21T22:25:04.878824+0000","last_unstale":"2024-01-21T22:25:07.297019+0000","last_undegraded":"2024-01-21T22:25:07.297019+0000","last_fullsized":"2024-01-21T22:25:07.297019+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:31:25.985811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304238+0000","last_change":"2024-01-21T22:25:04.886391+0000","last_active":"2024-01-21T22:25:07.304238+0000","last_peered":"2024-01-21T22:25:07.304238+0000","last_clean":"2024-01-21T22:25:07.304238+0000","last_became_active":"2024-01-21T22:25:04.884979+0000","last_became_peered":"2024-01-21T22:25:04.884979+0000","last_unstale":"2024-01-21T22:25:07.304238+0000","last_undegraded":"2024-01-21T22:25:07.304238+0000","last_fullsized":"2024-01-21T22:25:07.304238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:36:36.009632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286546+0000","last_change":"2024-01-21T22:25:04.888126+0000","last_active":"2024-01-21T22:25:07.286546+0000","last_peered":"2024-01-21T22:25:07.286546+0000","last_clean":"2024-01-21T22:25:07.286546+0000","last_became_active":"2024-01-21T22:25:04.887631+0000","last_became_peered":"2024-01-21T22:25:04.887631+0000","last_unstale":"2024-01-21T22:25:07.286546+0000","last_undegraded":"2024-01-21T22:25:07.286546+0000","last_fullsized":"2024-01-21T22:25:07.286546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:00:02.684745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286056+0000","last_change":"2024-01-21T22:25:04.885677+0000","last_active":"2024-01-21T22:25:07.286056+0000","last_peered":"2024-01-21T22:25:07.286056+0000","last_clean":"2024-01-21T22:25:07.286056+0000","last_became_active":"2024-01-21T22:25:04.884697+0000","last_became_peered":"2024-01-21T22:25:04.884697+0000","last_unstale":"2024-01-21T22:25:07.286056+0000","last_undegraded":"2024-01-21T22:25:07.286056+0000","last_fullsized":"2024-01-21T22:25:07.286056+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:44:23.749631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286124+0000","last_change":"2024-01-21T22:25:04.886799+0000","last_active":"2024-01-21T22:25:07.286124+0000","last_peered":"2024-01-21T22:25:07.286124+0000","last_clean":"2024-01-21T22:25:07.286124+0000","last_became_active":"2024-01-21T22:25:04.886387+0000","last_became_peered":"2024-01-21T22:25:04.886387+0000","last_unstale":"2024-01-21T22:25:07.286124+0000","last_undegraded":"2024-01-21T22:25:07.286124+0000","last_fullsized":"2024-01-21T22:25:07.286124+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:42:39.977125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304238+0000","last_change":"2024-01-21T22:25:04.888103+0000","last_active":"2024-01-21T22:25:07.304238+0000","last_peered":"2024-01-21T22:25:07.304238+0000","last_clean":"2024-01-21T22:25:07.304238+0000","last_became_active":"2024-01-21T22:25:04.887908+0000","last_became_peered":"2024-01-21T22:25:04.887908+0000","last_unstale":"2024-01-21T22:25:07.304238+0000","last_undegraded":"2024-01-21T22:25:07.304238+0000","last_fullsized":"2024-01-21T22:25:07.304238+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:23:52.444277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304247+0000","last_change":"2024-01-21T22:25:04.842750+0000","last_active":"2024-01-21T22:25:07.304247+0000","last_peered":"2024-01-21T22:25:07.304247+0000","last_clean":"2024-01-21T22:25:07.304247+0000","last_became_active":"2024-01-21T22:25:04.842512+0000","last_became_peered":"2024-01-21T22:25:04.842512+0000","last_unstale":"2024-01-21T22:25:07.304247+0000","last_undegraded":"2024-01-21T22:25:07.304247+0000","last_fullsized":"2024-01-21T22:25:07.304247+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:40:05.554704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.296486+0000","last_change":"2024-01-21T22:25:04.876592+0000","last_active":"2024-01-21T22:25:07.296486+0000","last_peered":"2024-01-21T22:25:07.296486+0000","last_clean":"2024-01-21T22:25:07.296486+0000","last_became_active":"2024-01-21T22:25:04.876305+0000","last_became_peered":"2024-01-21T22:25:04.876305+0000","last_unstale":"2024-01-21T22:25:07.296486+0000","last_undegraded":"2024-01-21T22:25:07.296486+0000","last_fullsized":"2024-01-21T22:25:07.296486+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:32.177562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093349+0000","last_change":"2024-01-21T22:25:04.878038+0000","last_active":"2024-01-21T22:25:07.093349+0000","last_peered":"2024-01-21T22:25:07.093349+0000","last_clean":"2024-01-21T22:25:07.093349+0000","last_became_active":"2024-01-21T22:25:04.877716+0000","last_became_peered":"2024-01-21T22:25:04.877716+0000","last_unstale":"2024-01-21T22:25:07.093349+0000","last_undegraded":"2024-01-21T22:25:07.093349+0000","last_fullsized":"2024-01-21T22:25:07.093349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:32:13.512213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.094445+0000","last_change":"2024-01-21T22:25:04.890537+0000","last_active":"2024-01-21T22:25:07.094445+0000","last_peered":"2024-01-21T22:25:07.094445+0000","last_clean":"2024-01-21T22:25:07.094445+0000","last_became_active":"2024-01-21T22:25:04.890298+0000","last_became_peered":"2024-01-21T22:25:04.890298+0000","last_unstale":"2024-01-21T22:25:07.094445+0000","last_undegraded":"2024-01-21T22:25:07.094445+0000","last_fullsized":"2024-01-21T22:25:07.094445+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:23:57.435028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.296659+0000","last_change":"2024-01-21T22:25:04.712868+0000","last_active":"2024-01-21T22:25:07.296659+0000","last_peered":"2024-01-21T22:25:07.296659+0000","last_clean":"2024-01-21T22:25:07.296659+0000","last_became_active":"2024-01-21T22:25:04.712616+0000","last_became_peered":"2024-01-21T22:25:04.712616+0000","last_unstale":"2024-01-21T22:25:07.296659+0000","last_undegraded":"2024-01-21T22:25:07.296659+0000","last_fullsized":"2024-01-21T22:25:07.296659+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:50:41.306582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.094344+0000","last_change":"2024-01-21T22:25:04.879328+0000","last_active":"2024-01-21T22:25:07.094344+0000","last_peered":"2024-01-21T22:25:07.094344+0000","last_clean":"2024-01-21T22:25:07.094344+0000","last_became_active":"2024-01-21T22:25:04.879011+0000","last_became_peered":"2024-01-21T22:25:04.879011+0000","last_unstale":"2024-01-21T22:25:07.094344+0000","last_undegraded":"2024-01-21T22:25:07.094344+0000","last_fullsized":"2024-01-21T22:25:07.094344+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:31:23.569672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286286+0000","last_change":"2024-01-21T22:25:04.885620+0000","last_active":"2024-01-21T22:25:07.286286+0000","last_peered":"2024-01-21T22:25:07.286286+0000","last_clean":"2024-01-21T22:25:07.286286+0000","last_became_active":"2024-01-21T22:25:04.884364+0000","last_became_peered":"2024-01-21T22:25:04.884364+0000","last_unstale":"2024-01-21T22:25:07.286286+0000","last_undegraded":"2024-01-21T22:25:07.286286+0000","last_fullsized":"2024-01-21T22:25:07.286286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:13:04.601911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151781+0000","last_change":"2024-01-21T22:25:04.868371+0000","last_active":"2024-01-21T22:25:07.151781+0000","last_peered":"2024-01-21T22:25:07.151781+0000","last_clean":"2024-01-21T22:25:07.151781+0000","last_became_active":"2024-01-21T22:25:04.868196+0000","last_became_peered":"2024-01-21T22:25:04.868196+0000","last_unstale":"2024-01-21T22:25:07.151781+0000","last_undegraded":"2024-01-21T22:25:07.151781+0000","last_fullsized":"2024-01-21T22:25:07.151781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:05:41.950232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.303520+0000","last_change":"2024-01-21T22:25:04.879144+0000","last_active":"2024-01-21T22:25:07.303520+0000","last_peered":"2024-01-21T22:25:07.303520+0000","last_clean":"2024-01-21T22:25:07.303520+0000","last_became_active":"2024-01-21T22:25:04.878868+0000","last_became_peered":"2024-01-21T22:25:04.878868+0000","last_unstale":"2024-01-21T22:25:07.303520+0000","last_undegraded":"2024-01-21T22:25:07.303520+0000","last_fullsized":"2024-01-21T22:25:07.303520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:55:28.069411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.296623+0000","last_change":"2024-01-21T22:25:04.715246+0000","last_active":"2024-01-21T22:25:07.296623+0000","last_peered":"2024-01-21T22:25:07.296623+0000","last_clean":"2024-01-21T22:25:07.296623+0000","last_became_active":"2024-01-21T22:25:04.714969+0000","last_became_peered":"2024-01-21T22:25:04.714969+0000","last_unstale":"2024-01-21T22:25:07.296623+0000","last_undegraded":"2024-01-21T22:25:07.296623+0000","last_fullsized":"2024-01-21T22:25:07.296623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:56:24.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093802+0000","last_change":"2024-01-21T22:25:04.877902+0000","last_active":"2024-01-21T22:25:07.093802+0000","last_peered":"2024-01-21T22:25:07.093802+0000","last_clean":"2024-01-21T22:25:07.093802+0000","last_became_active":"2024-01-21T22:25:04.876963+0000","last_became_peered":"2024-01-21T22:25:04.876963+0000","last_unstale":"2024-01-21T22:25:07.093802+0000","last_undegraded":"2024-01-21T22:25:07.093802+0000","last_fullsized":"2024-01-21T22:25:07.093802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:30:48.864454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297800+0000","last_change":"2024-01-21T22:25:04.716735+0000","last_active":"2024-01-21T22:25:07.297800+0000","last_peered":"2024-01-21T22:25:07.297800+0000","last_clean":"2024-01-21T22:25:07.297800+0000","last_became_active":"2024-01-21T22:25:04.716570+0000","last_became_peered":"2024-01-21T22:25:04.716570+0000","last_unstale":"2024-01-21T22:25:07.297800+0000","last_undegraded":"2024-01-21T22:25:07.297800+0000","last_fullsized":"2024-01-21T22:25:07.297800+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:44:55.158990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.143839+0000","last_change":"2024-01-21T22:25:04.886230+0000","last_active":"2024-01-21T22:25:07.143839+0000","last_peered":"2024-01-21T22:25:07.143839+0000","last_clean":"2024-01-21T22:25:07.143839+0000","last_became_active":"2024-01-21T22:25:04.885523+0000","last_became_peered":"2024-01-21T22:25:04.885523+0000","last_unstale":"2024-01-21T22:25:07.143839+0000","last_undegraded":"2024-01-21T22:25:07.143839+0000","last_fullsized":"2024-01-21T22:25:07.143839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:06:02.934920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304425+0000","last_change":"2024-01-21T22:25:04.842692+0000","last_active":"2024-01-21T22:25:07.304425+0000","last_peered":"2024-01-21T22:25:07.304425+0000","last_clean":"2024-01-21T22:25:07.304425+0000","last_became_active":"2024-01-21T22:25:04.842429+0000","last_became_peered":"2024-01-21T22:25:04.842429+0000","last_unstale":"2024-01-21T22:25:07.304425+0000","last_undegraded":"2024-01-21T22:25:07.304425+0000","last_fullsized":"2024-01-21T22:25:07.304425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:52:24.233709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304314+0000","last_change":"2024-01-21T22:25:04.888296+0000","last_active":"2024-01-21T22:25:07.304314+0000","last_peered":"2024-01-21T22:25:07.304314+0000","last_clean":"2024-01-21T22:25:07.304314+0000","last_became_active":"2024-01-21T22:25:04.888025+0000","last_became_peered":"2024-01-21T22:25:04.888025+0000","last_unstale":"2024-01-21T22:25:07.304314+0000","last_undegraded":"2024-01-21T22:25:07.304314+0000","last_fullsized":"2024-01-21T22:25:07.304314+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:57:44.129490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297099+0000","last_change":"2024-01-21T22:25:04.711908+0000","last_active":"2024-01-21T22:25:07.297099+0000","last_peered":"2024-01-21T22:25:07.297099+0000","last_clean":"2024-01-21T22:25:07.297099+0000","last_became_active":"2024-01-21T22:25:04.711654+0000","last_became_peered":"2024-01-21T22:25:04.711654+0000","last_unstale":"2024-01-21T22:25:07.297099+0000","last_undegraded":"2024-01-21T22:25:07.297099+0000","last_fullsized":"2024-01-21T22:25:07.297099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:20:41.612265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093463+0000","last_change":"2024-01-21T22:25:04.876757+0000","last_active":"2024-01-21T22:25:07.093463+0000","last_peered":"2024-01-21T22:25:07.093463+0000","last_clean":"2024-01-21T22:25:07.093463+0000","last_became_active":"2024-01-21T22:25:04.876012+0000","last_became_peered":"2024-01-21T22:25:04.876012+0000","last_unstale":"2024-01-21T22:25:07.093463+0000","last_undegraded":"2024-01-21T22:25:07.093463+0000","last_fullsized":"2024-01-21T22:25:07.093463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:00:56.239773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.296179+0000","last_change":"2024-01-21T22:25:04.715277+0000","last_active":"2024-01-21T22:25:07.296179+0000","last_peered":"2024-01-21T22:25:07.296179+0000","last_clean":"2024-01-21T22:25:07.296179+0000","last_became_active":"2024-01-21T22:25:04.715034+0000","last_became_peered":"2024-01-21T22:25:04.715034+0000","last_unstale":"2024-01-21T22:25:07.296179+0000","last_undegraded":"2024-01-21T22:25:07.296179+0000","last_fullsized":"2024-01-21T22:25:07.296179+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:55:28.069411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.143324+0000","last_change":"2024-01-21T22:25:04.844472+0000","last_active":"2024-01-21T22:25:07.143324+0000","last_peered":"2024-01-21T22:25:07.143324+0000","last_clean":"2024-01-21T22:25:07.143324+0000","last_became_active":"2024-01-21T22:25:04.843827+0000","last_became_peered":"2024-01-21T22:25:04.843827+0000","last_unstale":"2024-01-21T22:25:07.143324+0000","last_undegraded":"2024-01-21T22:25:07.143324+0000","last_fullsized":"2024-01-21T22:25:07.143324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:58:10.552987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.303755+0000","last_change":"2024-01-21T22:25:04.713764+0000","last_active":"2024-01-21T22:25:07.303755+0000","last_peered":"2024-01-21T22:25:07.303755+0000","last_clean":"2024-01-21T22:25:07.303755+0000","last_became_active":"2024-01-21T22:25:04.712814+0000","last_became_peered":"2024-01-21T22:25:04.712814+0000","last_unstale":"2024-01-21T22:25:07.303755+0000","last_undegraded":"2024-01-21T22:25:07.303755+0000","last_fullsized":"2024-01-21T22:25:07.303755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:16:41.440927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297178+0000","last_change":"2024-01-21T22:25:03.703079+0000","last_active":"2024-01-21T22:25:07.297178+0000","last_peered":"2024-01-21T22:25:07.297178+0000","last_clean":"2024-01-21T22:25:07.297178+0000","last_became_active":"2024-01-21T22:25:03.702776+0000","last_became_peered":"2024-01-21T22:25:03.702776+0000","last_unstale":"2024-01-21T22:25:07.297178+0000","last_undegraded":"2024-01-21T22:25:07.297178+0000","last_fullsized":"2024-01-21T22:25:07.297178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:34:13.302149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151769+0000","last_change":"2024-01-21T22:25:04.877158+0000","last_active":"2024-01-21T22:25:07.151769+0000","last_peered":"2024-01-21T22:25:07.151769+0000","last_clean":"2024-01-21T22:25:07.151769+0000","last_became_active":"2024-01-21T22:25:04.876920+0000","last_became_peered":"2024-01-21T22:25:04.876920+0000","last_unstale":"2024-01-21T22:25:07.151769+0000","last_undegraded":"2024-01-21T22:25:07.151769+0000","last_fullsized":"2024-01-21T22:25:07.151769+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:46:55.451173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093458+0000","last_change":"2024-01-21T22:25:03.720178+0000","last_active":"2024-01-21T22:25:07.093458+0000","last_peered":"2024-01-21T22:25:07.093458+0000","last_clean":"2024-01-21T22:25:07.093458+0000","last_became_active":"2024-01-21T22:25:03.719909+0000","last_became_peered":"2024-01-21T22:25:03.719909+0000","last_unstale":"2024-01-21T22:25:07.093458+0000","last_undegraded":"2024-01-21T22:25:07.093458+0000","last_fullsized":"2024-01-21T22:25:07.093458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:44:53.910872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.877462+0000","last_change":"2024-01-21T22:25:03.723005+0000","last_active":"2024-01-21T22:25:05.877462+0000","last_peered":"2024-01-21T22:25:05.877462+0000","last_clean":"2024-01-21T22:25:05.877462+0000","last_became_active":"2024-01-21T22:25:03.722758+0000","last_became_peered":"2024-01-21T22:25:03.722758+0000","last_unstale":"2024-01-21T22:25:05.877462+0000","last_undegraded":"2024-01-21T22:25:05.877462+0000","last_fullsized":"2024-01-21T22:25:05.877462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:16:40.192808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297215+0000","last_change":"2024-01-21T22:25:04.844030+0000","last_active":"2024-01-21T22:25:07.297215+0000","last_peered":"2024-01-21T22:25:07.297215+0000","last_clean":"2024-01-21T22:25:07.297215+0000","last_became_active":"2024-01-21T22:25:04.843714+0000","last_became_peered":"2024-01-21T22:25:04.843714+0000","last_unstale":"2024-01-21T22:25:07.297215+0000","last_undegraded":"2024-01-21T22:25:07.297215+0000","last_fullsized":"2024-01-21T22:25:07.297215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:25:50.380424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.521757+0000","last_change":"2024-01-21T22:25:03.702767+0000","last_active":"2024-01-21T22:25:05.521757+0000","last_peered":"2024-01-21T22:25:05.521757+0000","last_clean":"2024-01-21T22:25:05.521757+0000","last_became_active":"2024-01-21T22:25:03.702140+0000","last_became_peered":"2024-01-21T22:25:03.702140+0000","last_unstale":"2024-01-21T22:25:05.521757+0000","last_undegraded":"2024-01-21T22:25:05.521757+0000","last_fullsized":"2024-01-21T22:25:05.521757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:46:54.203055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151688+0000","last_change":"2024-01-21T22:25:04.867173+0000","last_active":"2024-01-21T22:25:07.151688+0000","last_peered":"2024-01-21T22:25:07.151688+0000","last_clean":"2024-01-21T22:25:07.151688+0000","last_became_active":"2024-01-21T22:25:04.866945+0000","last_became_peered":"2024-01-21T22:25:04.866945+0000","last_unstale":"2024-01-21T22:25:07.151688+0000","last_undegraded":"2024-01-21T22:25:07.151688+0000","last_fullsized":"2024-01-21T22:25:07.151688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:15:41.725132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.882129+0000","last_change":"2024-01-21T22:25:03.715128+0000","last_active":"2024-01-21T22:25:05.882129+0000","last_peered":"2024-01-21T22:25:05.882129+0000","last_clean":"2024-01-21T22:25:05.882129+0000","last_became_active":"2024-01-21T22:25:03.714039+0000","last_became_peered":"2024-01-21T22:25:03.714039+0000","last_unstale":"2024-01-21T22:25:05.882129+0000","last_undegraded":"2024-01-21T22:25:05.882129+0000","last_fullsized":"2024-01-21T22:25:05.882129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:09:09.988399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.144063+0000","last_change":"2024-01-21T22:25:04.886534+0000","last_active":"2024-01-21T22:25:07.144063+0000","last_peered":"2024-01-21T22:25:07.144063+0000","last_clean":"2024-01-21T22:25:07.144063+0000","last_became_active":"2024-01-21T22:25:04.885963+0000","last_became_peered":"2024-01-21T22:25:04.885963+0000","last_unstale":"2024-01-21T22:25:07.144063+0000","last_undegraded":"2024-01-21T22:25:07.144063+0000","last_fullsized":"2024-01-21T22:25:07.144063+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:44:31.867220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.610812+0000","last_change":"2024-01-21T22:25:03.718237+0000","last_active":"2024-01-21T22:25:05.610812+0000","last_peered":"2024-01-21T22:25:05.610812+0000","last_clean":"2024-01-21T22:25:05.610812+0000","last_became_active":"2024-01-21T22:25:03.717937+0000","last_became_peered":"2024-01-21T22:25:03.717937+0000","last_unstale":"2024-01-21T22:25:05.610812+0000","last_undegraded":"2024-01-21T22:25:05.610812+0000","last_fullsized":"2024-01-21T22:25:05.610812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:31:24.737692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298047+0000","last_change":"2024-01-21T22:25:04.882743+0000","last_active":"2024-01-21T22:25:07.298047+0000","last_peered":"2024-01-21T22:25:07.298047+0000","last_clean":"2024-01-21T22:25:07.298047+0000","last_became_active":"2024-01-21T22:25:04.882478+0000","last_became_peered":"2024-01-21T22:25:04.882478+0000","last_unstale":"2024-01-21T22:25:07.298047+0000","last_undegraded":"2024-01-21T22:25:07.298047+0000","last_fullsized":"2024-01-21T22:25:07.298047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:42:39.977125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.146198+0000","last_change":"2024-01-21T22:25:03.703672+0000","last_active":"2024-01-21T22:25:07.146198+0000","last_peered":"2024-01-21T22:25:07.146198+0000","last_clean":"2024-01-21T22:25:07.146198+0000","last_became_active":"2024-01-21T22:25:03.703426+0000","last_became_peered":"2024-01-21T22:25:03.703426+0000","last_unstale":"2024-01-21T22:25:07.146198+0000","last_undegraded":"2024-01-21T22:25:07.146198+0000","last_fullsized":"2024-01-21T22:25:07.146198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:41:51.793206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093763+0000","last_change":"2024-01-21T22:25:04.887306+0000","last_active":"2024-01-21T22:25:07.093763+0000","last_peered":"2024-01-21T22:25:07.093763+0000","last_clean":"2024-01-21T22:25:07.093763+0000","last_became_active":"2024-01-21T22:25:04.886970+0000","last_became_peered":"2024-01-21T22:25:04.886970+0000","last_unstale":"2024-01-21T22:25:07.093763+0000","last_undegraded":"2024-01-21T22:25:07.093763+0000","last_fullsized":"2024-01-21T22:25:07.093763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:59:45.239426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"23'6","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.158991+0000","last_change":"2024-01-21T22:25:03.714102+0000","last_active":"2024-01-21T22:25:07.158991+0000","last_peered":"2024-01-21T22:25:07.158991+0000","last_clean":"2024-01-21T22:25:07.158991+0000","last_became_active":"2024-01-21T22:25:03.713717+0000","last_became_peered":"2024-01-21T22:25:03.713717+0000","last_unstale":"2024-01-21T22:25:07.158991+0000","last_undegraded":"2024-01-21T22:25:07.158991+0000","last_fullsized":"2024-01-21T22:25:07.158991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:32:15.181238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286154+0000","last_change":"2024-01-21T22:25:04.881307+0000","last_active":"2024-01-21T22:25:07.286154+0000","last_peered":"2024-01-21T22:25:07.286154+0000","last_clean":"2024-01-21T22:25:07.286154+0000","last_became_active":"2024-01-21T22:25:04.880472+0000","last_became_peered":"2024-01-21T22:25:04.880472+0000","last_unstale":"2024-01-21T22:25:07.286154+0000","last_undegraded":"2024-01-21T22:25:07.286154+0000","last_fullsized":"2024-01-21T22:25:07.286154+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:29:02.950903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151339+0000","last_change":"2024-01-21T22:25:03.721825+0000","last_active":"2024-01-21T22:25:07.151339+0000","last_peered":"2024-01-21T22:25:07.151339+0000","last_clean":"2024-01-21T22:25:07.151339+0000","last_became_active":"2024-01-21T22:25:03.721502+0000","last_became_peered":"2024-01-21T22:25:03.721502+0000","last_unstale":"2024-01-21T22:25:07.151339+0000","last_undegraded":"2024-01-21T22:25:07.151339+0000","last_fullsized":"2024-01-21T22:25:07.151339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:09:09.988399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304218+0000","last_change":"2024-01-21T22:25:04.877455+0000","last_active":"2024-01-21T22:25:07.304218+0000","last_peered":"2024-01-21T22:25:07.304218+0000","last_clean":"2024-01-21T22:25:07.304218+0000","last_became_active":"2024-01-21T22:25:04.877257+0000","last_became_peered":"2024-01-21T22:25:04.877257+0000","last_unstale":"2024-01-21T22:25:07.304218+0000","last_undegraded":"2024-01-21T22:25:07.304218+0000","last_fullsized":"2024-01-21T22:25:07.304218+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:34:14.550267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.522179+0000","last_change":"2024-01-21T22:25:03.706077+0000","last_active":"2024-01-21T22:25:05.522179+0000","last_peered":"2024-01-21T22:25:05.522179+0000","last_clean":"2024-01-21T22:25:05.522179+0000","last_became_active":"2024-01-21T22:25:03.702798+0000","last_became_peered":"2024-01-21T22:25:03.702798+0000","last_unstale":"2024-01-21T22:25:05.522179+0000","last_undegraded":"2024-01-21T22:25:05.522179+0000","last_fullsized":"2024-01-21T22:25:05.522179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:20:40.364147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093643+0000","last_change":"2024-01-21T22:25:04.890636+0000","last_active":"2024-01-21T22:25:07.093643+0000","last_peered":"2024-01-21T22:25:07.093643+0000","last_clean":"2024-01-21T22:25:07.093643+0000","last_became_active":"2024-01-21T22:25:04.890271+0000","last_became_peered":"2024-01-21T22:25:04.890271+0000","last_unstale":"2024-01-21T22:25:07.093643+0000","last_undegraded":"2024-01-21T22:25:07.093643+0000","last_fullsized":"2024-01-21T22:25:07.093643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:15:46.754395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.609220+0000","last_change":"2024-01-21T22:25:03.717202+0000","last_active":"2024-01-21T22:25:05.609220+0000","last_peered":"2024-01-21T22:25:05.609220+0000","last_clean":"2024-01-21T22:25:05.609220+0000","last_became_active":"2024-01-21T22:25:03.716846+0000","last_became_peered":"2024-01-21T22:25:03.716846+0000","last_unstale":"2024-01-21T22:25:05.609220+0000","last_undegraded":"2024-01-21T22:25:05.609220+0000","last_fullsized":"2024-01-21T22:25:05.609220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:10.904923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.303888+0000","last_change":"2024-01-21T22:25:04.715053+0000","last_active":"2024-01-21T22:25:07.303888+0000","last_peered":"2024-01-21T22:25:07.303888+0000","last_clean":"2024-01-21T22:25:07.303888+0000","last_became_active":"2024-01-21T22:25:04.713566+0000","last_became_peered":"2024-01-21T22:25:04.713566+0000","last_unstale":"2024-01-21T22:25:07.303888+0000","last_undegraded":"2024-01-21T22:25:07.303888+0000","last_fullsized":"2024-01-21T22:25:07.303888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:50:41.306582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"23'8","reported_seq":22,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304071+0000","last_change":"2024-01-21T22:25:03.703017+0000","last_active":"2024-01-21T22:25:07.304071+0000","last_peered":"2024-01-21T22:25:07.304071+0000","last_clean":"2024-01-21T22:25:07.304071+0000","last_became_active":"2024-01-21T22:25:03.702742+0000","last_became_peered":"2024-01-21T22:25:03.702742+0000","last_unstale":"2024-01-21T22:25:07.304071+0000","last_undegraded":"2024-01-21T22:25:07.304071+0000","last_fullsized":"2024-01-21T22:25:07.304071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:42:03.386594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093975+0000","last_change":"2024-01-21T22:25:04.876507+0000","last_active":"2024-01-21T22:25:07.093975+0000","last_peered":"2024-01-21T22:25:07.093975+0000","last_clean":"2024-01-21T22:25:07.093975+0000","last_became_active":"2024-01-21T22:25:04.876169+0000","last_became_peered":"2024-01-21T22:25:04.876169+0000","last_unstale":"2024-01-21T22:25:07.093975+0000","last_undegraded":"2024-01-21T22:25:07.093975+0000","last_fullsized":"2024-01-21T22:25:07.093975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:12:11.958115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.152598+0000","last_change":"2024-01-21T22:25:03.720257+0000","last_active":"2024-01-21T22:25:07.152598+0000","last_peered":"2024-01-21T22:25:07.152598+0000","last_clean":"2024-01-21T22:25:07.152598+0000","last_became_active":"2024-01-21T22:25:03.720009+0000","last_became_peered":"2024-01-21T22:25:03.720009+0000","last_unstale":"2024-01-21T22:25:07.152598+0000","last_undegraded":"2024-01-21T22:25:07.152598+0000","last_fullsized":"2024-01-21T22:25:07.152598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:37:56.569677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.144048+0000","last_change":"2024-01-21T22:25:04.883112+0000","last_active":"2024-01-21T22:25:07.144048+0000","last_peered":"2024-01-21T22:25:07.144048+0000","last_clean":"2024-01-21T22:25:07.144048+0000","last_became_active":"2024-01-21T22:25:04.882767+0000","last_became_peered":"2024-01-21T22:25:04.882767+0000","last_unstale":"2024-01-21T22:25:07.144048+0000","last_undegraded":"2024-01-21T22:25:07.144048+0000","last_fullsized":"2024-01-21T22:25:07.144048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:18:03.439738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.303803+0000","last_change":"2024-01-21T22:25:03.703516+0000","last_active":"2024-01-21T22:25:07.303803+0000","last_peered":"2024-01-21T22:25:07.303803+0000","last_clean":"2024-01-21T22:25:07.303803+0000","last_became_active":"2024-01-21T22:25:03.703256+0000","last_became_peered":"2024-01-21T22:25:03.703256+0000","last_unstale":"2024-01-21T22:25:07.303803+0000","last_undegraded":"2024-01-21T22:25:07.303803+0000","last_fullsized":"2024-01-21T22:25:07.303803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:57:42.881371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286834+0000","last_change":"2024-01-21T22:25:04.884129+0000","last_active":"2024-01-21T22:25:07.286834+0000","last_peered":"2024-01-21T22:25:07.286834+0000","last_clean":"2024-01-21T22:25:07.286834+0000","last_became_active":"2024-01-21T22:25:04.883839+0000","last_became_peered":"2024-01-21T22:25:04.883839+0000","last_unstale":"2024-01-21T22:25:07.286834+0000","last_undegraded":"2024-01-21T22:25:07.286834+0000","last_fullsized":"2024-01-21T22:25:07.286834+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:43:55.049706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298049+0000","last_change":"2024-01-21T22:25:03.708366+0000","last_active":"2024-01-21T22:25:07.298049+0000","last_peered":"2024-01-21T22:25:07.298049+0000","last_clean":"2024-01-21T22:25:07.298049+0000","last_became_active":"2024-01-21T22:25:03.708101+0000","last_became_peered":"2024-01-21T22:25:03.708101+0000","last_unstale":"2024-01-21T22:25:07.298049+0000","last_undegraded":"2024-01-21T22:25:07.298049+0000","last_fullsized":"2024-01-21T22:25:07.298049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:57:42.881371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.286513+0000","last_change":"2024-01-21T22:25:04.877993+0000","last_active":"2024-01-21T22:25:07.286513+0000","last_peered":"2024-01-21T22:25:07.286513+0000","last_clean":"2024-01-21T22:25:07.286513+0000","last_became_active":"2024-01-21T22:25:04.877661+0000","last_became_peered":"2024-01-21T22:25:04.877661+0000","last_unstale":"2024-01-21T22:25:07.286513+0000","last_undegraded":"2024-01-21T22:25:07.286513+0000","last_fullsized":"2024-01-21T22:25:07.286513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:20:40.947205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.878348+0000","last_change":"2024-01-21T22:25:03.721559+0000","last_active":"2024-01-21T22:25:05.878348+0000","last_peered":"2024-01-21T22:25:05.878348+0000","last_clean":"2024-01-21T22:25:05.878348+0000","last_became_active":"2024-01-21T22:25:03.721281+0000","last_became_peered":"2024-01-21T22:25:03.721281+0000","last_unstale":"2024-01-21T22:25:05.878348+0000","last_undegraded":"2024-01-21T22:25:05.878348+0000","last_fullsized":"2024-01-21T22:25:05.878348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:55:19.679388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.094279+0000","last_change":"2024-01-21T22:25:04.883087+0000","last_active":"2024-01-21T22:25:07.094279+0000","last_peered":"2024-01-21T22:25:07.094279+0000","last_clean":"2024-01-21T22:25:07.094279+0000","last_became_active":"2024-01-21T22:25:04.880227+0000","last_became_peered":"2024-01-21T22:25:04.880227+0000","last_unstale":"2024-01-21T22:25:07.094279+0000","last_undegraded":"2024-01-21T22:25:07.094279+0000","last_fullsized":"2024-01-21T22:25:07.094279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:58:09.653538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.880999+0000","last_change":"2024-01-21T22:24:52.968324+0000","last_active":"2024-01-21T22:25:05.880999+0000","last_peered":"2024-01-21T22:25:05.880999+0000","last_clean":"2024-01-21T22:25:05.880999+0000","last_became_active":"2024-01-21T22:24:52.967018+0000","last_became_peered":"2024-01-21T22:24:52.967018+0000","last_unstale":"2024-01-21T22:25:05.880999+0000","last_undegraded":"2024-01-21T22:25:05.880999+0000","last_fullsized":"2024-01-21T22:25:05.880999+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:24:50.312540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:24:50.312540+0000","last_clean_scrub_stamp":"2024-01-21T22:24:50.312540+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:52:04.800073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.522445+0000","last_change":"2024-01-21T22:25:03.707407+0000","last_active":"2024-01-21T22:25:05.522445+0000","last_peered":"2024-01-21T22:25:05.522445+0000","last_clean":"2024-01-21T22:25:05.522445+0000","last_became_active":"2024-01-21T22:25:03.707085+0000","last_became_peered":"2024-01-21T22:25:03.707085+0000","last_unstale":"2024-01-21T22:25:05.522445+0000","last_undegraded":"2024-01-21T22:25:05.522445+0000","last_fullsized":"2024-01-21T22:25:05.522445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:44:22.501512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151755+0000","last_change":"2024-01-21T22:25:04.868564+0000","last_active":"2024-01-21T22:25:07.151755+0000","last_peered":"2024-01-21T22:25:07.151755+0000","last_clean":"2024-01-21T22:25:07.151755+0000","last_became_active":"2024-01-21T22:25:04.868247+0000","last_became_peered":"2024-01-21T22:25:04.868247+0000","last_unstale":"2024-01-21T22:25:07.151755+0000","last_undegraded":"2024-01-21T22:25:07.151755+0000","last_fullsized":"2024-01-21T22:25:07.151755+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:30:50.039501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.521811+0000","last_change":"2024-01-21T22:25:03.706070+0000","last_active":"2024-01-21T22:25:05.521811+0000","last_peered":"2024-01-21T22:25:05.521811+0000","last_clean":"2024-01-21T22:25:05.521811+0000","last_became_active":"2024-01-21T22:25:03.705785+0000","last_became_peered":"2024-01-21T22:25:03.705785+0000","last_unstale":"2024-01-21T22:25:05.521811+0000","last_undegraded":"2024-01-21T22:25:05.521811+0000","last_fullsized":"2024-01-21T22:25:05.521811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:30.929444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304132+0000","last_change":"2024-01-21T22:25:04.845436+0000","last_active":"2024-01-21T22:25:07.304132+0000","last_peered":"2024-01-21T22:25:07.304132+0000","last_clean":"2024-01-21T22:25:07.304132+0000","last_became_active":"2024-01-21T22:25:04.845274+0000","last_became_peered":"2024-01-21T22:25:04.845274+0000","last_unstale":"2024-01-21T22:25:07.304132+0000","last_undegraded":"2024-01-21T22:25:07.304132+0000","last_fullsized":"2024-01-21T22:25:07.304132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:08:36.686787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.878258+0000","last_change":"2024-01-21T22:25:03.709174+0000","last_active":"2024-01-21T22:25:05.878258+0000","last_peered":"2024-01-21T22:25:05.878258+0000","last_clean":"2024-01-21T22:25:05.878258+0000","last_became_active":"2024-01-21T22:25:03.708894+0000","last_became_peered":"2024-01-21T22:25:03.708894+0000","last_unstale":"2024-01-21T22:25:05.878258+0000","last_undegraded":"2024-01-21T22:25:05.878258+0000","last_fullsized":"2024-01-21T22:25:05.878258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:19:26.591140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297083+0000","last_change":"2024-01-21T22:25:04.711947+0000","last_active":"2024-01-21T22:25:07.297083+0000","last_peered":"2024-01-21T22:25:07.297083+0000","last_clean":"2024-01-21T22:25:07.297083+0000","last_became_active":"2024-01-21T22:25:04.711694+0000","last_became_peered":"2024-01-21T22:25:04.711694+0000","last_unstale":"2024-01-21T22:25:07.297083+0000","last_undegraded":"2024-01-21T22:25:07.297083+0000","last_fullsized":"2024-01-21T22:25:07.297083+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:37:57.817795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.522504+0000","last_change":"2024-01-21T22:25:03.706355+0000","last_active":"2024-01-21T22:25:05.522504+0000","last_peered":"2024-01-21T22:25:05.522504+0000","last_clean":"2024-01-21T22:25:05.522504+0000","last_became_active":"2024-01-21T22:25:03.703876+0000","last_became_peered":"2024-01-21T22:25:03.703876+0000","last_unstale":"2024-01-21T22:25:05.522504+0000","last_undegraded":"2024-01-21T22:25:05.522504+0000","last_fullsized":"2024-01-21T22:25:05.522504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:42:38.729006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.285835+0000","last_change":"2024-01-21T22:25:04.885997+0000","last_active":"2024-01-21T22:25:07.285835+0000","last_peered":"2024-01-21T22:25:07.285835+0000","last_clean":"2024-01-21T22:25:07.285835+0000","last_became_active":"2024-01-21T22:25:04.884162+0000","last_became_peered":"2024-01-21T22:25:04.884162+0000","last_unstale":"2024-01-21T22:25:07.285835+0000","last_undegraded":"2024-01-21T22:25:07.285835+0000","last_fullsized":"2024-01-21T22:25:07.285835+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:44:55.158990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.521383+0000","last_change":"2024-01-21T22:25:03.708300+0000","last_active":"2024-01-21T22:25:05.521383+0000","last_peered":"2024-01-21T22:25:05.521383+0000","last_clean":"2024-01-21T22:25:05.521383+0000","last_became_active":"2024-01-21T22:25:03.708036+0000","last_became_peered":"2024-01-21T22:25:03.708036+0000","last_unstale":"2024-01-21T22:25:05.521383+0000","last_undegraded":"2024-01-21T22:25:05.521383+0000","last_fullsized":"2024-01-21T22:25:05.521383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:23:51.196159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.143828+0000","last_change":"2024-01-21T22:25:04.844713+0000","last_active":"2024-01-21T22:25:07.143828+0000","last_peered":"2024-01-21T22:25:07.143828+0000","last_clean":"2024-01-21T22:25:07.143828+0000","last_became_active":"2024-01-21T22:25:04.844008+0000","last_became_peered":"2024-01-21T22:25:04.844008+0000","last_unstale":"2024-01-21T22:25:07.143828+0000","last_undegraded":"2024-01-21T22:25:07.143828+0000","last_fullsized":"2024-01-21T22:25:07.143828+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:33:25.760064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.523113+0000","last_change":"2024-01-21T22:25:03.699093+0000","last_active":"2024-01-21T22:25:05.523113+0000","last_peered":"2024-01-21T22:25:05.523113+0000","last_clean":"2024-01-21T22:25:05.523113+0000","last_became_active":"2024-01-21T22:25:03.698443+0000","last_became_peered":"2024-01-21T22:25:03.698443+0000","last_unstale":"2024-01-21T22:25:05.523113+0000","last_undegraded":"2024-01-21T22:25:05.523113+0000","last_fullsized":"2024-01-21T22:25:05.523113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:20:39.699087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304411+0000","last_change":"2024-01-21T22:25:04.842733+0000","last_active":"2024-01-21T22:25:07.304411+0000","last_peered":"2024-01-21T22:25:07.304411+0000","last_clean":"2024-01-21T22:25:07.304411+0000","last_became_active":"2024-01-21T22:25:04.842479+0000","last_became_peered":"2024-01-21T22:25:04.842479+0000","last_unstale":"2024-01-21T22:25:07.304411+0000","last_undegraded":"2024-01-21T22:25:07.304411+0000","last_fullsized":"2024-01-21T22:25:07.304411+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:51:06.931790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151980+0000","last_change":"2024-01-21T22:25:03.720306+0000","last_active":"2024-01-21T22:25:07.151980+0000","last_peered":"2024-01-21T22:25:07.151980+0000","last_clean":"2024-01-21T22:25:07.151980+0000","last_became_active":"2024-01-21T22:25:03.720067+0000","last_became_peered":"2024-01-21T22:25:03.720067+0000","last_unstale":"2024-01-21T22:25:07.151980+0000","last_undegraded":"2024-01-21T22:25:07.151980+0000","last_fullsized":"2024-01-21T22:25:07.151980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:52:22.985590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151977+0000","last_change":"2024-01-21T22:25:04.859628+0000","last_active":"2024-01-21T22:25:07.151977+0000","last_peered":"2024-01-21T22:25:07.151977+0000","last_clean":"2024-01-21T22:25:07.151977+0000","last_became_active":"2024-01-21T22:25:04.859270+0000","last_became_peered":"2024-01-21T22:25:04.859270+0000","last_unstale":"2024-01-21T22:25:07.151977+0000","last_undegraded":"2024-01-21T22:25:07.151977+0000","last_fullsized":"2024-01-21T22:25:07.151977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:56:24.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.282102+0000","last_change":"2024-01-21T22:25:03.722006+0000","last_active":"2024-01-21T22:25:07.282102+0000","last_peered":"2024-01-21T22:25:07.282102+0000","last_clean":"2024-01-21T22:25:07.282102+0000","last_became_active":"2024-01-21T22:25:03.721275+0000","last_became_peered":"2024-01-21T22:25:03.721275+0000","last_unstale":"2024-01-21T22:25:07.282102+0000","last_undegraded":"2024-01-21T22:25:07.282102+0000","last_fullsized":"2024-01-21T22:25:07.282102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:43:53.801587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151916+0000","last_change":"2024-01-21T22:25:04.858864+0000","last_active":"2024-01-21T22:25:07.151916+0000","last_peered":"2024-01-21T22:25:07.151916+0000","last_clean":"2024-01-21T22:25:07.151916+0000","last_became_active":"2024-01-21T22:25:04.858586+0000","last_became_peered":"2024-01-21T22:25:04.858586+0000","last_unstale":"2024-01-21T22:25:07.151916+0000","last_undegraded":"2024-01-21T22:25:07.151916+0000","last_fullsized":"2024-01-21T22:25:07.151916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:32.177562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.609941+0000","last_change":"2024-01-21T22:25:03.718436+0000","last_active":"2024-01-21T22:25:05.609941+0000","last_peered":"2024-01-21T22:25:05.609941+0000","last_clean":"2024-01-21T22:25:05.609941+0000","last_became_active":"2024-01-21T22:25:03.718171+0000","last_became_peered":"2024-01-21T22:25:03.718171+0000","last_unstale":"2024-01-21T22:25:05.609941+0000","last_undegraded":"2024-01-21T22:25:05.609941+0000","last_fullsized":"2024-01-21T22:25:05.609941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:05:40.702113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093711+0000","last_change":"2024-01-21T22:25:04.875736+0000","last_active":"2024-01-21T22:25:07.093711+0000","last_peered":"2024-01-21T22:25:07.093711+0000","last_clean":"2024-01-21T22:25:07.093711+0000","last_became_active":"2024-01-21T22:25:04.875368+0000","last_became_peered":"2024-01-21T22:25:04.875368+0000","last_unstale":"2024-01-21T22:25:07.093711+0000","last_undegraded":"2024-01-21T22:25:07.093711+0000","last_fullsized":"2024-01-21T22:25:07.093711+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:39:31.273250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.877906+0000","last_change":"2024-01-21T22:25:03.722820+0000","last_active":"2024-01-21T22:25:05.877906+0000","last_peered":"2024-01-21T22:25:05.877906+0000","last_clean":"2024-01-21T22:25:05.877906+0000","last_became_active":"2024-01-21T22:25:03.722525+0000","last_became_peered":"2024-01-21T22:25:03.722525+0000","last_unstale":"2024-01-21T22:25:05.877906+0000","last_undegraded":"2024-01-21T22:25:05.877906+0000","last_fullsized":"2024-01-21T22:25:05.877906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:40:04.306585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.151604+0000","last_change":"2024-01-21T22:25:04.857451+0000","last_active":"2024-01-21T22:25:07.151604+0000","last_peered":"2024-01-21T22:25:07.151604+0000","last_clean":"2024-01-21T22:25:07.151604+0000","last_became_active":"2024-01-21T22:25:04.857205+0000","last_became_peered":"2024-01-21T22:25:04.857205+0000","last_unstale":"2024-01-21T22:25:07.151604+0000","last_undegraded":"2024-01-21T22:25:07.151604+0000","last_fullsized":"2024-01-21T22:25:07.151604+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:12.153042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:05.877746+0000","last_change":"2024-01-21T22:25:03.708574+0000","last_active":"2024-01-21T22:25:05.877746+0000","last_peered":"2024-01-21T22:25:05.877746+0000","last_clean":"2024-01-21T22:25:05.877746+0000","last_became_active":"2024-01-21T22:25:03.708318+0000","last_became_peered":"2024-01-21T22:25:03.708318+0000","last_unstale":"2024-01-21T22:25:05.877746+0000","last_undegraded":"2024-01-21T22:25:05.877746+0000","last_fullsized":"2024-01-21T22:25:05.877746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:50:40.058463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.296286+0000","last_change":"2024-01-21T22:25:04.843079+0000","last_active":"2024-01-21T22:25:07.296286+0000","last_peered":"2024-01-21T22:25:07.296286+0000","last_clean":"2024-01-21T22:25:07.296286+0000","last_became_active":"2024-01-21T22:25:04.842815+0000","last_became_peered":"2024-01-21T22:25:04.842815+0000","last_unstale":"2024-01-21T22:25:07.296286+0000","last_undegraded":"2024-01-21T22:25:07.296286+0000","last_fullsized":"2024-01-21T22:25:07.296286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:16:41.440927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"23'2","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.281616+0000","last_change":"2024-01-21T22:25:03.710506+0000","last_active":"2024-01-21T22:25:07.281616+0000","last_peered":"2024-01-21T22:25:07.281616+0000","last_clean":"2024-01-21T22:25:07.281616+0000","last_became_active":"2024-01-21T22:25:03.710262+0000","last_became_peered":"2024-01-21T22:25:03.710262+0000","last_unstale":"2024-01-21T22:25:07.281616+0000","last_undegraded":"2024-01-21T22:25:07.281616+0000","last_fullsized":"2024-01-21T22:25:07.281616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:25:49.132306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.093352+0000","last_change":"2024-01-21T22:25:04.876469+0000","last_active":"2024-01-21T22:25:07.093352+0000","last_peered":"2024-01-21T22:25:07.093352+0000","last_clean":"2024-01-21T22:25:07.093352+0000","last_became_active":"2024-01-21T22:25:04.875619+0000","last_became_peered":"2024-01-21T22:25:04.875619+0000","last_unstale":"2024-01-21T22:25:07.093352+0000","last_undegraded":"2024-01-21T22:25:07.093352+0000","last_fullsized":"2024-01-21T22:25:07.093352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:30:20.167607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.094159+0000","last_change":"2024-01-21T22:25:04.891067+0000","last_active":"2024-01-21T22:25:07.094159+0000","last_peered":"2024-01-21T22:25:07.094159+0000","last_clean":"2024-01-21T22:25:07.094159+0000","last_became_active":"2024-01-21T22:25:04.890515+0000","last_became_peered":"2024-01-21T22:25:04.890515+0000","last_unstale":"2024-01-21T22:25:07.094159+0000","last_undegraded":"2024-01-21T22:25:07.094159+0000","last_fullsized":"2024-01-21T22:25:07.094159+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:25:48.726591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.158438+0000","last_change":"2024-01-21T22:25:03.719412+0000","last_active":"2024-01-21T22:25:07.158438+0000","last_peered":"2024-01-21T22:25:07.158438+0000","last_clean":"2024-01-21T22:25:07.158438+0000","last_became_active":"2024-01-21T22:25:03.718163+0000","last_became_peered":"2024-01-21T22:25:03.718163+0000","last_unstale":"2024-01-21T22:25:07.158438+0000","last_undegraded":"2024-01-21T22:25:07.158438+0000","last_fullsized":"2024-01-21T22:25:07.158438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:56:22.817420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.304192+0000","last_change":"2024-01-21T22:25:04.889166+0000","last_active":"2024-01-21T22:25:07.304192+0000","last_peered":"2024-01-21T22:25:07.304192+0000","last_clean":"2024-01-21T22:25:07.304192+0000","last_became_active":"2024-01-21T22:25:04.888910+0000","last_became_peered":"2024-01-21T22:25:04.888910+0000","last_unstale":"2024-01-21T22:25:07.304192+0000","last_undegraded":"2024-01-21T22:25:07.304192+0000","last_fullsized":"2024-01-21T22:25:07.304192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:55:53.892439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.156719+0000","last_change":"2024-01-21T22:25:03.718932+0000","last_active":"2024-01-21T22:25:07.156719+0000","last_peered":"2024-01-21T22:25:07.156719+0000","last_clean":"2024-01-21T22:25:07.156719+0000","last_became_active":"2024-01-21T22:25:03.718649+0000","last_became_peered":"2024-01-21T22:25:03.718649+0000","last_unstale":"2024-01-21T22:25:07.156719+0000","last_undegraded":"2024-01-21T22:25:07.156719+0000","last_fullsized":"2024-01-21T22:25:07.156719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:30.929444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.297922+0000","last_change":"2024-01-21T22:25:04.715429+0000","last_active":"2024-01-21T22:25:07.297922+0000","last_peered":"2024-01-21T22:25:07.297922+0000","last_clean":"2024-01-21T22:25:07.297922+0000","last_became_active":"2024-01-21T22:25:04.714720+0000","last_became_peered":"2024-01-21T22:25:04.714720+0000","last_unstale":"2024-01-21T22:25:07.297922+0000","last_undegraded":"2024-01-21T22:25:07.297922+0000","last_fullsized":"2024-01-21T22:25:07.297922+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:44:23.749631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298298+0000","last_change":"2024-01-21T22:25:03.703289+0000","last_active":"2024-01-21T22:25:07.298298+0000","last_peered":"2024-01-21T22:25:07.298298+0000","last_clean":"2024-01-21T22:25:07.298298+0000","last_became_active":"2024-01-21T22:25:03.703020+0000","last_became_peered":"2024-01-21T22:25:03.703020+0000","last_unstale":"2024-01-21T22:25:07.298298+0000","last_undegraded":"2024-01-21T22:25:07.298298+0000","last_fullsized":"2024-01-21T22:25:07.298298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:10.904923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.298304+0000","last_change":"2024-01-21T22:25:04.715894+0000","last_active":"2024-01-21T22:25:07.298304+0000","last_peered":"2024-01-21T22:25:07.298304+0000","last_clean":"2024-01-21T22:25:07.298304+0000","last_became_active":"2024-01-21T22:25:04.715652+0000","last_became_peered":"2024-01-21T22:25:04.715652+0000","last_unstale":"2024-01-21T22:25:07.298304+0000","last_undegraded":"2024-01-21T22:25:07.298304+0000","last_fullsized":"2024-01-21T22:25:07.298304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:13:04.601911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"23'4","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-01-21T22:25:07.303733+0000","last_change":"2024-01-21T22:25:03.702339+0000","last_active":"2024-01-21T22:25:07.303733+0000","last_peered":"2024-01-21T22:25:07.303733+0000","last_clean":"2024-01-21T22:25:07.303733+0000","last_became_active":"2024-01-21T22:25:03.702087+0000","last_became_peered":"2024-01-21T22:25:03.702087+0000","last_unstale":"2024-01-21T22:25:07.303733+0000","last_undegraded":"2024-01-21T22:25:07.303733+0000","last_fullsized":"2024-01-21T22:25:07.303733+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:51:05.683671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542156,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":245760,"data_stored":85152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]}]},{"osd":6,"up_from":14,"seq":60129542156,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":85028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542156,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":839680,"data_stored":675488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542156,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":87312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]}]},{"osd":3,"up_from":14,"seq":60129542157,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":241664,"data_stored":85610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5489999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3660000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.488}]}]},{"osd":2,"up_from":14,"seq":60129542155,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":249856,"data_stored":86640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94599999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542156,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":835584,"data_stored":675466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.28}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.097}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4610000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542157,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":245760,"data_stored":85084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.458}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2470000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0309999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2769999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-21T22:25:26.306 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-01-21T22:25:26.306 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-01-21T22:25:26.306 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-01-21T22:25:26.307 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-01-21T22:25:26.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.547+0000 7f23cb000640 1 Processor -- start 2024-01-21T22:25:26.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.547+0000 7f23cb000640 1 -- start start 2024-01-21T22:25:26.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.549+0000 7f23cb000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f23c41519a0 0x7f23c4071960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:26.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.549+0000 7f23cb000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23c406c310 0x7f23c4157790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:26.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.549+0000 7f23cb000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c4157cd0 con 0x7f23c40c5ab0 2024-01-21T22:25:26.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.549+0000 7f23cb000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c406d9f0 con 0x7f23c41519a0 2024-01-21T22:25:26.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.549+0000 7f23cb000640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c406d150 con 0x7f23c406c310 2024-01-21T22:25:26.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.550+0000 7f23c3e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f23c41519a0 0x7f23c4071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:26.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.550+0000 7f23bb200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f23c40c5ab0 0x7f23c40cb2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:53638/0 (socket says 172.21.15.138:53638) 2024-01-21T22:25:26.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.550+0000 7f23bb200640 1 -- 172.21.15.138:0/4294322931 learned_addr learned my addr 172.21.15.138:0/4294322931 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:26.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.553+0000 7f23c9000640 1 --2- 172.21.15.138:0/4294322931 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23c406c310 0x7f23c4157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:26.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.555+0000 7f23c3e00640 1 -- 172.21.15.138:0/4294322931 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23c406c310 msgr2=0x7f23c4157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:26.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.555+0000 7f23c3e00640 1 --2- 172.21.15.138:0/4294322931 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23c406c310 0x7f23c4157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:26.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.555+0000 7f23c3e00640 1 -- 172.21.15.138:0/4294322931 >> v1:172.21.15.152:6789/0 conn(0x7f23c40c5ab0 legacy=0x7f23c40cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:26.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.555+0000 7f23c3e00640 1 -- 172.21.15.138:0/4294322931 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c406d430 con 0x7f23c41519a0 2024-01-21T22:25:26.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.557+0000 7f23c3e00640 1 --2- 172.21.15.138:0/4294322931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c41519a0 0x7f23c4071960 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0097a0 tx=0x7f23ac02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba07e95b52b612f6 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:26.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.561+0000 7f23c3400640 1 -- 172.21.15.138:0/4294322931 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac02d040 con 0x7f23c41519a0 2024-01-21T22:25:26.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.561+0000 7f23c3400640 1 -- 172.21.15.138:0/4294322931 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ac031810 con 0x7f23c41519a0 2024-01-21T22:25:26.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.561+0000 7f23c3400640 1 -- 172.21.15.138:0/4294322931 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac031ba0 con 0x7f23c41519a0 2024-01-21T22:25:26.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.561+0000 7f23cb000640 1 -- 172.21.15.138:0/4294322931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c41519a0 msgr2=0x7f23c4071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:26.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.561+0000 7f23cb000640 1 --2- 172.21.15.138:0/4294322931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c41519a0 0x7f23c4071960 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0097a0 tx=0x7f23ac02b8b0 comp rx=0 tx=0).stop 2024-01-21T22:25:26.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.562+0000 7f23cb000640 1 -- 172.21.15.138:0/4294322931 shutdown_connections 2024-01-21T22:25:26.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.562+0000 7f23cb000640 1 --2- 172.21.15.138:0/4294322931 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23c406c310 0x7f23c4157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:26.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.562+0000 7f23cb000640 1 --2- 172.21.15.138:0/4294322931 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c41519a0 0x7f23c4071960 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:26.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.562+0000 7f23cb000640 1 -- 172.21.15.138:0/4294322931 >> 172.21.15.138:0/4294322931 conn(0x7f23c406a8d0 msgr2=0x7f23c406ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:26.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.562+0000 7f23cb000640 1 -- 172.21.15.138:0/4294322931 shutdown_connections 2024-01-21T22:25:26.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.562+0000 7f23cb000640 1 -- 172.21.15.138:0/4294322931 wait complete. 2024-01-21T22:25:26.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.564+0000 7f23cb000640 1 Processor -- start 2024-01-21T22:25:26.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.564+0000 7f23cb000640 1 -- start start 2024-01-21T22:25:26.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.565+0000 7f23cb000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c406c310 0x7f23c41eb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:26.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.565+0000 7f23cb000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23c40c5ab0 0x7f23c41573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:26.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.565+0000 7f23cb000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23c41519a0 0x7f23c4155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:26.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.565+0000 7f23cb000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c4155970 con 0x7f23c41519a0 2024-01-21T22:25:26.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.565+0000 7f23cb000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c414ee10 con 0x7f23c406c310 2024-01-21T22:25:26.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.565+0000 7f23cb000640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c414f0f0 con 0x7f23c40c5ab0 2024-01-21T22:25:26.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.565+0000 7f23c3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c406c310 0x7f23c41eb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:26.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.565+0000 7f23bb200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23c40c5ab0 0x7f23c41573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:26.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.566+0000 7f23c3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c406c310 0x7f23c41eb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:36606/0 (socket says 172.21.15.138:36606) 2024-01-21T22:25:26.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.566+0000 7f23c3e00640 1 -- 172.21.15.138:0/1762033050 learned_addr learned my addr 172.21.15.138:0/1762033050 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:26.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.569+0000 7f23bb200640 1 -- 172.21.15.138:0/1762033050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c406c310 msgr2=0x7f23c41eb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:26.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.569+0000 7f23bb200640 1 --2- 172.21.15.138:0/1762033050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c406c310 0x7f23c41eb8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:26.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.569+0000 7f23bb200640 1 -- 172.21.15.138:0/1762033050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23c41519a0 msgr2=0x7f23c4155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:26.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.569+0000 7f23bb200640 1 --2- 172.21.15.138:0/1762033050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23c41519a0 0x7f23c4155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:26.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.569+0000 7f23bb200640 1 -- 172.21.15.138:0/1762033050 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ac009400 con 0x7f23c40c5ab0 2024-01-21T22:25:26.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.571+0000 7f23bb200640 1 --2- 172.21.15.138:0/1762033050 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23c40c5ab0 0x7f23c41573d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f23b4000f60 tx=0x7f23b400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:26.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.574+0000 7f23c0c00640 1 -- 172.21.15.138:0/1762033050 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b4005ce0 con 0x7f23c40c5ab0 2024-01-21T22:25:26.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.575+0000 7f23c0c00640 1 -- 172.21.15.138:0/1762033050 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b4005e40 con 0x7f23c40c5ab0 2024-01-21T22:25:26.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.575+0000 7f23c0c00640 1 -- 172.21.15.138:0/1762033050 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b4007310 con 0x7f23c40c5ab0 2024-01-21T22:25:26.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.575+0000 7f23cb000640 1 -- 172.21.15.138:0/1762033050 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c414f3d0 con 0x7f23c40c5ab0 2024-01-21T22:25:26.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.575+0000 7f23cb000640 1 -- 172.21.15.138:0/1762033050 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c4155600 con 0x7f23c40c5ab0 2024-01-21T22:25:26.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.576+0000 7f23cb000640 1 -- 172.21.15.138:0/1762033050 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c41557c0 con 0x7f23c40c5ab0 2024-01-21T22:25:26.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.597+0000 7f23c0c00640 1 -- 172.21.15.138:0/1762033050 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f23b4007470 con 0x7f23c40c5ab0 2024-01-21T22:25:26.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.599+0000 7f23c0c00640 1 --2- 172.21.15.138:0/1762033050 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f239c06b3b0 0x7f239c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:26.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.599+0000 7f23c0c00640 1 -- 172.21.15.138:0/1762033050 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23b4089370 con 0x7f23c40c5ab0 2024-01-21T22:25:26.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.600+0000 7f23c3e00640 1 --2- 172.21.15.138:0/1762033050 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f239c06b3b0 0x7f239c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:26.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.600+0000 7f23c3e00640 1 --2- 172.21.15.138:0/1762033050 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f239c06b3b0 0x7f239c06d870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f23ac02b8f0 tx=0x7f23ac03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:26.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:26.835+0000 7f23c0c00640 1 -- 172.21.15.138:0/1762033050 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f23b4054190 con 0x7f23c40c5ab0 2024-01-21T22:25:27.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.088+0000 7f23cb000640 1 -- 172.21.15.138:0/1762033050 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f23c4155ae0 con 0x7f23c40c5ab0 2024-01-21T22:25:27.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.093+0000 7f23c0c00640 1 -- 172.21.15.138:0/1762033050 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f23b40577b0 con 0x7f23c40c5ab0 2024-01-21T22:25:27.094 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:25:27.094 INFO:teuthology.orchestra.run.smithi138.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-01-21T22:25:27.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.095+0000 7f23cb000640 1 -- 172.21.15.138:0/1762033050 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f239c06b3b0 msgr2=0x7f239c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:27.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.095+0000 7f23cb000640 1 --2- 172.21.15.138:0/1762033050 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f239c06b3b0 0x7f239c06d870 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f23ac02b8f0 tx=0x7f23ac03c040 comp rx=0 tx=0).stop 2024-01-21T22:25:27.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.095+0000 7f23cb000640 1 -- 172.21.15.138:0/1762033050 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23c40c5ab0 msgr2=0x7f23c41573d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:27.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.095+0000 7f23cb000640 1 --2- 172.21.15.138:0/1762033050 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23c40c5ab0 0x7f23c41573d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f23b4000f60 tx=0x7f23b400c800 comp rx=0 tx=0).stop 2024-01-21T22:25:27.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.096+0000 7f23cb000640 1 -- 172.21.15.138:0/1762033050 shutdown_connections 2024-01-21T22:25:27.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.096+0000 7f23cb000640 1 --2- 172.21.15.138:0/1762033050 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f239c06b3b0 0x7f239c06d870 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:27.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.096+0000 7f23cb000640 1 --2- 172.21.15.138:0/1762033050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23c41519a0 0x7f23c4155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:27.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.096+0000 7f23cb000640 1 --2- 172.21.15.138:0/1762033050 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f23c40c5ab0 0x7f23c41573d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:27.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.096+0000 7f23cb000640 1 --2- 172.21.15.138:0/1762033050 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f23c406c310 0x7f23c41eb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:27.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.096+0000 7f23cb000640 1 -- 172.21.15.138:0/1762033050 >> 172.21.15.138:0/1762033050 conn(0x7f23c406a8d0 msgr2=0x7f23c40c2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:27.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.096+0000 7f23cb000640 1 -- 172.21.15.138:0/1762033050 shutdown_connections 2024-01-21T22:25:27.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.096+0000 7f23cb000640 1 -- 172.21.15.138:0/1762033050 wait complete. 2024-01-21T22:25:27.117 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-01-21T22:25:27.117 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-21T22:25:27.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.325+0000 7ffabca00640 1 Processor -- start 2024-01-21T22:25:27.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.325+0000 7ffabca00640 1 -- start start 2024-01-21T22:25:27.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.327+0000 7ffabca00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffab4154ed0 0x7ffab406a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:27.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.327+0000 7ffabca00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ffab406c2a0 0x7ffab4157d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:27.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.327+0000 7ffabca00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab415be20 con 0x7ffab40c9700 2024-01-21T22:25:27.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.327+0000 7ffabca00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab406b420 con 0x7ffab4154ed0 2024-01-21T22:25:27.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.327+0000 7ffabca00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab406ac30 con 0x7ffab406c2a0 2024-01-21T22:25:27.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.328+0000 7ffaba000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffab4154ed0 0x7ffab406a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:27.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.328+0000 7ffaba000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ffab4154ed0 0x7ffab406a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53302/0 (socket says 172.21.15.138:53302) 2024-01-21T22:25:27.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.328+0000 7ffaba000640 1 -- 172.21.15.138:0/2299507170 learned_addr learned my addr 172.21.15.138:0/2299507170 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:27.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.328+0000 7ffabaa00640 1 --2- 172.21.15.138:0/2299507170 >> v2:172.21.15.152:3300/0 conn(0x7ffab406c2a0 0x7ffab4157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:27.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.331+0000 7ffaba000640 1 -- 172.21.15.138:0/2299507170 >> v2:172.21.15.152:3300/0 conn(0x7ffab406c2a0 msgr2=0x7ffab4157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:27.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.331+0000 7ffaba000640 1 --2- 172.21.15.138:0/2299507170 >> v2:172.21.15.152:3300/0 conn(0x7ffab406c2a0 0x7ffab4157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:27.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.331+0000 7ffaba000640 1 -- 172.21.15.138:0/2299507170 >> v1:172.21.15.152:6789/0 conn(0x7ffab40c9700 legacy=0x7ffab40c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:27.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.331+0000 7ffaba000640 1 -- 172.21.15.138:0/2299507170 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab415af30 con 0x7ffab4154ed0 2024-01-21T22:25:27.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.333+0000 7ffaba000640 1 --2- 172.21.15.138:0/2299507170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffab4154ed0 0x7ffab406a6f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ffab40c3010 tx=0x7ffaa402c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ab7ba191d6ba87 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:27.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.338+0000 7ffab8c00640 1 -- 172.21.15.138:0/2299507170 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa402cbb0 con 0x7ffab4154ed0 2024-01-21T22:25:27.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.338+0000 7ffab8c00640 1 -- 172.21.15.138:0/2299507170 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa402cd10 con 0x7ffab4154ed0 2024-01-21T22:25:27.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.338+0000 7ffab8c00640 1 -- 172.21.15.138:0/2299507170 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4034610 con 0x7ffab4154ed0 2024-01-21T22:25:27.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.339+0000 7ffabca00640 1 -- 172.21.15.138:0/2299507170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffab4154ed0 msgr2=0x7ffab406a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:27.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.339+0000 7ffabca00640 1 --2- 172.21.15.138:0/2299507170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffab4154ed0 0x7ffab406a6f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ffab40c3010 tx=0x7ffaa402c010 comp rx=0 tx=0).stop 2024-01-21T22:25:27.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.339+0000 7ffabca00640 1 -- 172.21.15.138:0/2299507170 shutdown_connections 2024-01-21T22:25:27.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.339+0000 7ffabca00640 1 --2- 172.21.15.138:0/2299507170 >> v2:172.21.15.152:3300/0 conn(0x7ffab406c2a0 0x7ffab4157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:27.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.339+0000 7ffabca00640 1 --2- 172.21.15.138:0/2299507170 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffab4154ed0 0x7ffab406a6f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:27.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.339+0000 7ffabca00640 1 -- 172.21.15.138:0/2299507170 >> 172.21.15.138:0/2299507170 conn(0x7ffab4066a90 msgr2=0x7ffab40722f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:27.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.339+0000 7ffabca00640 1 -- 172.21.15.138:0/2299507170 shutdown_connections 2024-01-21T22:25:27.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.340+0000 7ffabca00640 1 -- 172.21.15.138:0/2299507170 wait complete. 2024-01-21T22:25:27.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.341+0000 7ffabca00640 1 Processor -- start 2024-01-21T22:25:27.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.341+0000 7ffabca00640 1 -- start start 2024-01-21T22:25:27.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.342+0000 7ffabca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffab406c2a0 0x7ffab40c47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:27.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.342+0000 7ffabca00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffab40c9700 0x7ffab41f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:27.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.342+0000 7ffabca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffab4154ed0 0x7ffab41487e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:27.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.342+0000 7ffabca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab41579b0 con 0x7ffab406c2a0 2024-01-21T22:25:27.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.342+0000 7ffabca00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab40c1470 con 0x7ffab4154ed0 2024-01-21T22:25:27.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.342+0000 7ffabca00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab40c7a80 con 0x7ffab40c9700 2024-01-21T22:25:27.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.343+0000 7ffabaa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffab4154ed0 0x7ffab41487e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:27.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.343+0000 7ffaba000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffab406c2a0 0x7ffab40c47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:27.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.343+0000 7ffab9600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffab40c9700 0x7ffab41f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:27.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.343+0000 7ffabaa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffab4154ed0 0x7ffab41487e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:36636/0 (socket says 172.21.15.138:36636) 2024-01-21T22:25:27.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.343+0000 7ffabaa00640 1 -- 172.21.15.138:0/3888712963 learned_addr learned my addr 172.21.15.138:0/3888712963 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:27.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.347+0000 7ffabaa00640 1 -- 172.21.15.138:0/3888712963 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffab40c9700 msgr2=0x7ffab41f2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:27.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.347+0000 7ffabaa00640 1 --2- 172.21.15.138:0/3888712963 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffab40c9700 0x7ffab41f2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:27.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.347+0000 7ffabaa00640 1 -- 172.21.15.138:0/3888712963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffab406c2a0 msgr2=0x7ffab40c47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:27.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.347+0000 7ffabaa00640 1 --2- 172.21.15.138:0/3888712963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffab406c2a0 0x7ffab40c47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:27.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.347+0000 7ffabaa00640 1 -- 172.21.15.138:0/3888712963 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa4009b30 con 0x7ffab4154ed0 2024-01-21T22:25:27.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.349+0000 7ffabaa00640 1 --2- 172.21.15.138:0/3888712963 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffab4154ed0 0x7ffab41487e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00e960 tx=0x7ffaac00ee20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:27.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.351+0000 7ffab2000640 1 -- 172.21.15.138:0/3888712963 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac009690 con 0x7ffab4154ed0 2024-01-21T22:25:27.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.351+0000 7ffabca00640 1 -- 172.21.15.138:0/3888712963 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab41573f0 con 0x7ffab4154ed0 2024-01-21T22:25:27.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.351+0000 7ffabca00640 1 -- 172.21.15.138:0/3888712963 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab40c7430 con 0x7ffab4154ed0 2024-01-21T22:25:27.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.352+0000 7ffabca00640 1 -- 172.21.15.138:0/3888712963 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab40c7290 con 0x7ffab4154ed0 2024-01-21T22:25:27.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.353+0000 7ffab2000640 1 -- 172.21.15.138:0/3888712963 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaac005990 con 0x7ffab4154ed0 2024-01-21T22:25:27.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.353+0000 7ffab2000640 1 -- 172.21.15.138:0/3888712963 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac005d00 con 0x7ffab4154ed0 2024-01-21T22:25:27.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.620+0000 7ffab2000640 1 -- 172.21.15.138:0/3888712963 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7ffaac00c920 con 0x7ffab4154ed0 2024-01-21T22:25:27.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.622+0000 7ffab2000640 1 --2- 172.21.15.138:0/3888712963 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7ffa8c06b330 0x7ffa8c06d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:27.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.623+0000 7ffaba000640 1 --2- 172.21.15.138:0/3888712963 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7ffa8c06b330 0x7ffa8c06d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:27.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.624+0000 7ffaba000640 1 --2- 172.21.15.138:0/3888712963 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7ffa8c06b330 0x7ffa8c06d7f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4035000 tx=0x7ffaa4030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:27.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.630+0000 7ffab2000640 1 -- 172.21.15.138:0/3888712963 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffaac058d10 con 0x7ffab4154ed0 2024-01-21T22:25:27.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.640+0000 7ffab2000640 1 -- 172.21.15.138:0/3888712963 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7ffaac055050 con 0x7ffab4154ed0 2024-01-21T22:25:27.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.859+0000 7ffabca00640 1 -- 172.21.15.138:0/3888712963 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffab40c7290 con 0x7ffab4154ed0 2024-01-21T22:25:27.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:27.997+0000 7ffab2000640 1 -- 172.21.15.138:0/3888712963 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16522 (secure 0 0 0) 0x7ffab40c7290 con 0x7ffab4154ed0 2024-01-21T22:25:27.999 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:25:27.999 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":23,"fsid":"d488d446-149f-48d0-89ca-b2830ae72846","created":"2024-01-21T22:24:28.581932+0000","modified":"2024-01-21T22:25:06.475790+0000","last_up_change":"2024-01-21T22:24:47.145799+0000","last_in_change":"2024-01-21T22:24:37.672807+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-21T22:24:49.586178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-21T22:25:02.044454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-21T22:25:03.361765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d812b2d6-c8a3-4015-bfdb-5766b25eae5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6825","nonce":2849203277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6827","nonce":2849203277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6831","nonce":2849203277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6829","nonce":2849203277}]},"public_addr":"172.21.15.138:6825/2849203277","cluster_addr":"172.21.15.138:6827/2849203277","heartbeat_back_addr":"172.21.15.138:6831/2849203277","heartbeat_front_addr":"172.21.15.138:6829/2849203277","state":["exists","up"]},{"osd":1,"uuid":"d8d875df-c840-46cc-b6c0-a847f9c93999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6817","nonce":174521731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6819","nonce":174521731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6823","nonce":174521731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6821","nonce":174521731}]},"public_addr":"172.21.15.138:6817/174521731","cluster_addr":"172.21.15.138:6819/174521731","heartbeat_back_addr":"172.21.15.138:6823/174521731","heartbeat_front_addr":"172.21.15.138:6821/174521731","state":["exists","up"]},{"osd":2,"uuid":"4bd94373-76ca-47a6-870b-16d96d7c9e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6801","nonce":1365838748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6803","nonce":1365838748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6809","nonce":1365838748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6805","nonce":1365838748}]},"public_addr":"172.21.15.138:6801/1365838748","cluster_addr":"172.21.15.138:6803/1365838748","heartbeat_back_addr":"172.21.15.138:6809/1365838748","heartbeat_front_addr":"172.21.15.138:6805/1365838748","state":["exists","up"]},{"osd":3,"uuid":"07267087-d4d2-4e88-ab9b-bb87f6dd061a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6808","nonce":1841873885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6811","nonce":1841873885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6815","nonce":1841873885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6813","nonce":1841873885}]},"public_addr":"172.21.15.138:6808/1841873885","cluster_addr":"172.21.15.138:6811/1841873885","heartbeat_back_addr":"172.21.15.138:6815/1841873885","heartbeat_front_addr":"172.21.15.138:6813/1841873885","state":["exists","up"]},{"osd":4,"uuid":"b47ef758-d783-4168-a9a5-11db3de4ca12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6827","nonce":3960812298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6829","nonce":3960812298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6833","nonce":3960812298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6831","nonce":3960812298}]},"public_addr":"172.21.15.152:6827/3960812298","cluster_addr":"172.21.15.152:6829/3960812298","heartbeat_back_addr":"172.21.15.152:6833/3960812298","heartbeat_front_addr":"172.21.15.152:6831/3960812298","state":["exists","up"]},{"osd":5,"uuid":"0ea5d43e-61d8-4823-815f-9e42c62fae97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6819","nonce":4095234314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6821","nonce":4095234314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6825","nonce":4095234314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6823","nonce":4095234314}]},"public_addr":"172.21.15.152:6819/4095234314","cluster_addr":"172.21.15.152:6821/4095234314","heartbeat_back_addr":"172.21.15.152:6825/4095234314","heartbeat_front_addr":"172.21.15.152:6823/4095234314","state":["exists","up"]},{"osd":6,"uuid":"59ffac36-e2e0-4693-ac82-ad5eccf6224d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6803","nonce":223848127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6805","nonce":223848127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6809","nonce":223848127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6807","nonce":223848127}]},"public_addr":"172.21.15.152:6803/223848127","cluster_addr":"172.21.15.152:6805/223848127","heartbeat_back_addr":"172.21.15.152:6809/223848127","heartbeat_front_addr":"172.21.15.152:6807/223848127","state":["exists","up"]},{"osd":7,"uuid":"4f892c7d-32a5-4960-bf88-4ed7c863cd0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6811","nonce":52509095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6813","nonce":52509095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6817","nonce":52509095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6815","nonce":52509095}]},"public_addr":"172.21.15.152:6811/52509095","cluster_addr":"172.21.15.152:6813/52509095","heartbeat_back_addr":"172.21.15.152:6817/52509095","heartbeat_front_addr":"172.21.15.152:6815/52509095","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.434460+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.374709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.613134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.298148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.242144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.198102+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.093385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.197180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-21T22:25:28.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.001+0000 7ffabca00640 1 -- 172.21.15.138:0/3888712963 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7ffa8c06b330 msgr2=0x7ffa8c06d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:28.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.001+0000 7ffabca00640 1 --2- 172.21.15.138:0/3888712963 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7ffa8c06b330 0x7ffa8c06d7f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4035000 tx=0x7ffaa4030000 comp rx=0 tx=0).stop 2024-01-21T22:25:28.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.001+0000 7ffabca00640 1 -- 172.21.15.138:0/3888712963 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffab4154ed0 msgr2=0x7ffab41487e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:28.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.001+0000 7ffabca00640 1 --2- 172.21.15.138:0/3888712963 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffab4154ed0 0x7ffab41487e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00e960 tx=0x7ffaac00ee20 comp rx=0 tx=0).stop 2024-01-21T22:25:28.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.001+0000 7ffabca00640 1 -- 172.21.15.138:0/3888712963 shutdown_connections 2024-01-21T22:25:28.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.001+0000 7ffabca00640 1 --2- 172.21.15.138:0/3888712963 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7ffa8c06b330 0x7ffa8c06d7f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:28.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.001+0000 7ffabca00640 1 --2- 172.21.15.138:0/3888712963 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ffab4154ed0 0x7ffab41487e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:28.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.001+0000 7ffabca00640 1 --2- 172.21.15.138:0/3888712963 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ffab40c9700 0x7ffab41f2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:28.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.001+0000 7ffabca00640 1 --2- 172.21.15.138:0/3888712963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffab406c2a0 0x7ffab40c47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:28.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.001+0000 7ffabca00640 1 -- 172.21.15.138:0/3888712963 >> 172.21.15.138:0/3888712963 conn(0x7ffab4066a90 msgr2=0x7ffab4150c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:28.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.001+0000 7ffabca00640 1 -- 172.21.15.138:0/3888712963 shutdown_connections 2024-01-21T22:25:28.003 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.001+0000 7ffabca00640 1 -- 172.21.15.138:0/3888712963 wait complete. 2024-01-21T22:25:28.022 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-21T22:24:49.586178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-21T22:25:02.044454+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-21T22:25:03.361765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-21T22:25:28.023 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-21T22:25:28.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.232+0000 7f1566800640 1 Processor -- start 2024-01-21T22:25:28.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.232+0000 7f1566800640 1 -- start start 2024-01-21T22:25:28.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.234+0000 7f1566800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f15601583e0 0x7f1560070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:28.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.234+0000 7f1566800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f15600c5b90 0x7f156006b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:28.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.234+0000 7f1566800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15601622c0 con 0x7f1560158000 2024-01-21T22:25:28.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.234+0000 7f1566800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1560161460 con 0x7f15601583e0 2024-01-21T22:25:28.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.234+0000 7f1566800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1560161740 con 0x7f15600c5b90 2024-01-21T22:25:28.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.235+0000 7f155fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f15600c5b90 0x7f156006b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:28.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.235+0000 7f155ea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f15601583e0 0x7f1560070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:28.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.235+0000 7f155fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f15600c5b90 0x7f156006b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:51316/0 (socket says 172.21.15.138:51316) 2024-01-21T22:25:28.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.235+0000 7f155fe00640 1 -- 172.21.15.138:0/1857882505 learned_addr learned my addr 172.21.15.138:0/1857882505 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:28.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.237+0000 7f155fe00640 1 -- 172.21.15.138:0/1857882505 >> v2:172.21.15.138:3300/0 conn(0x7f15601583e0 msgr2=0x7f1560070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:28.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.237+0000 7f155fe00640 1 --2- 172.21.15.138:0/1857882505 >> v2:172.21.15.138:3300/0 conn(0x7f15601583e0 0x7f1560070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:28.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.237+0000 7f155fe00640 1 -- 172.21.15.138:0/1857882505 >> v1:172.21.15.138:6789/0 conn(0x7f1560158000 legacy=0x7f15600c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:28.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.237+0000 7f155fe00640 1 -- 172.21.15.138:0/1857882505 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1560161a20 con 0x7f15600c5b90 2024-01-21T22:25:28.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.240+0000 7f155fe00640 1 --2- 172.21.15.138:0/1857882505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15600c5b90 0x7f156006b480 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f15600c61b0 tx=0x7f155802ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ad111c4bc874f1b server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:28.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.244+0000 7f155e000640 1 -- 172.21.15.138:0/1857882505 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155802f850 con 0x7f15600c5b90 2024-01-21T22:25:28.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.244+0000 7f155e000640 1 -- 172.21.15.138:0/1857882505 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155802f9b0 con 0x7f15600c5b90 2024-01-21T22:25:28.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.244+0000 7f155e000640 1 -- 172.21.15.138:0/1857882505 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155802fd40 con 0x7f15600c5b90 2024-01-21T22:25:28.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.244+0000 7f1566800640 1 -- 172.21.15.138:0/1857882505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15600c5b90 msgr2=0x7f156006b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:28.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.244+0000 7f1566800640 1 --2- 172.21.15.138:0/1857882505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15600c5b90 0x7f156006b480 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f15600c61b0 tx=0x7f155802ba30 comp rx=0 tx=0).stop 2024-01-21T22:25:28.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.245+0000 7f1566800640 1 -- 172.21.15.138:0/1857882505 shutdown_connections 2024-01-21T22:25:28.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.245+0000 7f1566800640 1 --2- 172.21.15.138:0/1857882505 >> v2:172.21.15.138:3300/0 conn(0x7f15601583e0 0x7f1560070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:28.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.245+0000 7f1566800640 1 --2- 172.21.15.138:0/1857882505 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15600c5b90 0x7f156006b480 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:28.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.245+0000 7f1566800640 1 -- 172.21.15.138:0/1857882505 >> 172.21.15.138:0/1857882505 conn(0x7f1560072a20 msgr2=0x7f1560066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:28.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.245+0000 7f1566800640 1 -- 172.21.15.138:0/1857882505 shutdown_connections 2024-01-21T22:25:28.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.245+0000 7f1566800640 1 -- 172.21.15.138:0/1857882505 wait complete. 2024-01-21T22:25:28.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.247+0000 7f1566800640 1 Processor -- start 2024-01-21T22:25:28.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.247+0000 7f1566800640 1 -- start start 2024-01-21T22:25:28.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.249+0000 7f1566800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15600c5b90 0x7f156015d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:28.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.249+0000 7f1566800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1560158000 0x7f15601598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:28.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.249+0000 7f1566800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15601583e0 0x7f15601f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:28.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.249+0000 7f1566800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1560159e20 con 0x7f15600c5b90 2024-01-21T22:25:28.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.249+0000 7f1566800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15601f7e00 con 0x7f15601583e0 2024-01-21T22:25:28.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.249+0000 7f1566800640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156015dd30 con 0x7f1560158000 2024-01-21T22:25:28.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.249+0000 7f155f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15600c5b90 0x7f156015d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:28.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.250+0000 7f155fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15601583e0 0x7f15601f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:28.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.250+0000 7f155ea00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1560158000 0x7f15601598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:28.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.250+0000 7f155f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15600c5b90 0x7f156015d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40100/0 (socket says 172.21.15.138:40100) 2024-01-21T22:25:28.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.250+0000 7f155f400640 1 -- 172.21.15.138:0/1125763478 learned_addr learned my addr 172.21.15.138:0/1125763478 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:28.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.253+0000 7f155fe00640 1 -- 172.21.15.138:0/1125763478 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1560158000 msgr2=0x7f15601598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:28.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.253+0000 7f155fe00640 1 --2- 172.21.15.138:0/1125763478 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1560158000 0x7f15601598e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:28.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.253+0000 7f155fe00640 1 -- 172.21.15.138:0/1125763478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15600c5b90 msgr2=0x7f156015d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:28.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.253+0000 7f155fe00640 1 --2- 172.21.15.138:0/1125763478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15600c5b90 0x7f156015d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:28.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.253+0000 7f155fe00640 1 -- 172.21.15.138:0/1125763478 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15580094d0 con 0x7f15601583e0 2024-01-21T22:25:28.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.255+0000 7f155fe00640 1 --2- 172.21.15.138:0/1125763478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15601583e0 0x7f15601f78c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1560161880 tx=0x7f1558004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:28.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.259+0000 7f1557400640 1 -- 172.21.15.138:0/1125763478 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1558037c40 con 0x7f15601583e0 2024-01-21T22:25:28.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.259+0000 7f1557400640 1 -- 172.21.15.138:0/1125763478 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1558037dc0 con 0x7f15601583e0 2024-01-21T22:25:28.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.259+0000 7f1557400640 1 -- 172.21.15.138:0/1125763478 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15580426c0 con 0x7f15601583e0 2024-01-21T22:25:28.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.259+0000 7f1566800640 1 -- 172.21.15.138:0/1125763478 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15601f3ae0 con 0x7f15601583e0 2024-01-21T22:25:28.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.260+0000 7f1566800640 1 -- 172.21.15.138:0/1125763478 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156015da50 con 0x7f15601583e0 2024-01-21T22:25:28.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.261+0000 7f1566800640 1 -- 172.21.15.138:0/1125763478 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15601f3940 con 0x7f15601583e0 2024-01-21T22:25:28.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.281+0000 7f1557400640 1 -- 172.21.15.138:0/1125763478 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f155804c440 con 0x7f15601583e0 2024-01-21T22:25:28.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.282+0000 7f1557400640 1 --2- 172.21.15.138:0/1125763478 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f153806b380 0x7f153806d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:28.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.282+0000 7f155f400640 1 --2- 172.21.15.138:0/1125763478 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f153806b380 0x7f153806d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:28.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.283+0000 7f155f400640 1 --2- 172.21.15.138:0/1125763478 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f153806b380 0x7f153806d840 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1548001840 tx=0x7f1548009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:28.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.508+0000 7f1557400640 1 -- 172.21.15.138:0/1125763478 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f155803c070 con 0x7f15601583e0 2024-01-21T22:25:28.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.520+0000 7f1557400640 1 -- 172.21.15.138:0/1125763478 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f1558077160 con 0x7f15601583e0 2024-01-21T22:25:28.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.721+0000 7f1566800640 1 -- 172.21.15.138:0/1125763478 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15601f3940 con 0x7f15601583e0 2024-01-21T22:25:28.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.742+0000 7f1557400640 1 -- 172.21.15.138:0/1125763478 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f15601f3940 con 0x7f15601583e0 2024-01-21T22:25:28.743 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-01-21T22:25:28.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.745+0000 7f1566800640 1 -- 172.21.15.138:0/1125763478 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f153806b380 msgr2=0x7f153806d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:28.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.745+0000 7f1566800640 1 --2- 172.21.15.138:0/1125763478 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f153806b380 0x7f153806d840 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1548001840 tx=0x7f1548009040 comp rx=0 tx=0).stop 2024-01-21T22:25:28.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.745+0000 7f1566800640 1 -- 172.21.15.138:0/1125763478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15601583e0 msgr2=0x7f15601f78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:28.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.745+0000 7f1566800640 1 --2- 172.21.15.138:0/1125763478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15601583e0 0x7f15601f78c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1560161880 tx=0x7f1558004730 comp rx=0 tx=0).stop 2024-01-21T22:25:28.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.745+0000 7f1566800640 1 -- 172.21.15.138:0/1125763478 shutdown_connections 2024-01-21T22:25:28.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.745+0000 7f1566800640 1 --2- 172.21.15.138:0/1125763478 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f153806b380 0x7f153806d840 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:28.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.745+0000 7f1566800640 1 --2- 172.21.15.138:0/1125763478 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15601583e0 0x7f15601f78c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:28.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.745+0000 7f1566800640 1 --2- 172.21.15.138:0/1125763478 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1560158000 0x7f15601598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:28.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.745+0000 7f1566800640 1 --2- 172.21.15.138:0/1125763478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15600c5b90 0x7f156015d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:28.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.745+0000 7f1566800640 1 -- 172.21.15.138:0/1125763478 >> 172.21.15.138:0/1125763478 conn(0x7f1560072a20 msgr2=0x7f156015e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:28.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.745+0000 7f1566800640 1 -- 172.21.15.138:0/1125763478 shutdown_connections 2024-01-21T22:25:28.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.745+0000 7f1566800640 1 -- 172.21.15.138:0/1125763478 wait complete. 2024-01-21T22:25:28.765 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-21T22:25:28.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.967+0000 7ff163c00640 1 Processor -- start 2024-01-21T22:25:28.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.968+0000 7ff163c00640 1 -- start start 2024-01-21T22:25:28.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.969+0000 7ff163c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff15c15e000 0x7ff15c0c6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:28.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.969+0000 7ff163c00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7ff15c0c9300 0x7ff15c0cc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:28.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.969+0000 7ff163c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff15c0c8e70 con 0x7ff15c158560 2024-01-21T22:25:28.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.969+0000 7ff163c00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff15c0c8930 con 0x7ff15c0c9300 2024-01-21T22:25:28.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.969+0000 7ff163c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff15c0c8b40 con 0x7ff15c15e000 2024-01-21T22:25:28.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.970+0000 7ff15be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff15c15e000 0x7ff15c0c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:28.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.970+0000 7ff15be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7ff15c15e000 0x7ff15c0c6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:51334/0 (socket says 172.21.15.138:51334) 2024-01-21T22:25:28.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.970+0000 7ff15be00640 1 -- 172.21.15.138:0/1476909174 learned_addr learned my addr 172.21.15.138:0/1476909174 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:28.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.972+0000 7ff15be00640 1 -- 172.21.15.138:0/1476909174 >> v2:172.21.15.138:3300/0 conn(0x7ff15c0c9300 msgr2=0x7ff15c0cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:28.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.972+0000 7ff15be00640 1 --2- 172.21.15.138:0/1476909174 >> v2:172.21.15.138:3300/0 conn(0x7ff15c0c9300 0x7ff15c0cc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:28.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.972+0000 7ff15be00640 1 -- 172.21.15.138:0/1476909174 >> v1:172.21.15.138:6789/0 conn(0x7ff15c158560 legacy=0x7ff15c0c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:28.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.972+0000 7ff15be00640 1 -- 172.21.15.138:0/1476909174 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15c06a890 con 0x7ff15c15e000 2024-01-21T22:25:28.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.974+0000 7ff15be00640 1 --2- 172.21.15.138:0/1476909174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff15c15e000 0x7ff15c0c6910 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0c32d0 tx=0x7ff14402bf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fec2f852e9721873 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:28.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.978+0000 7ff15b400640 1 -- 172.21.15.138:0/1476909174 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14402cb20 con 0x7ff15c15e000 2024-01-21T22:25:28.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.978+0000 7ff15b400640 1 -- 172.21.15.138:0/1476909174 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14402cc80 con 0x7ff15c15e000 2024-01-21T22:25:28.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.978+0000 7ff15b400640 1 -- 172.21.15.138:0/1476909174 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1440343f0 con 0x7ff15c15e000 2024-01-21T22:25:28.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.979+0000 7ff163c00640 1 -- 172.21.15.138:0/1476909174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff15c15e000 msgr2=0x7ff15c0c6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:28.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.979+0000 7ff163c00640 1 --2- 172.21.15.138:0/1476909174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff15c15e000 0x7ff15c0c6910 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0c32d0 tx=0x7ff14402bf80 comp rx=0 tx=0).stop 2024-01-21T22:25:28.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.979+0000 7ff163c00640 1 -- 172.21.15.138:0/1476909174 shutdown_connections 2024-01-21T22:25:28.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.979+0000 7ff163c00640 1 --2- 172.21.15.138:0/1476909174 >> v2:172.21.15.138:3300/0 conn(0x7ff15c0c9300 0x7ff15c0cc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:28.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.979+0000 7ff163c00640 1 --2- 172.21.15.138:0/1476909174 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff15c15e000 0x7ff15c0c6910 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:28.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.979+0000 7ff163c00640 1 -- 172.21.15.138:0/1476909174 >> 172.21.15.138:0/1476909174 conn(0x7ff15c066400 msgr2=0x7ff15c0667e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:28.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.979+0000 7ff163c00640 1 -- 172.21.15.138:0/1476909174 shutdown_connections 2024-01-21T22:25:28.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.979+0000 7ff163c00640 1 -- 172.21.15.138:0/1476909174 wait complete. 2024-01-21T22:25:28.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.981+0000 7ff163c00640 1 Processor -- start 2024-01-21T22:25:28.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.981+0000 7ff163c00640 1 -- start start 2024-01-21T22:25:28.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.982+0000 7ff163c00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff15c0c9300 0x7ff15c1f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:28.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.982+0000 7ff163c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff15c158560 0x7ff15c153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:28.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.982+0000 7ff163c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff15c15e000 0x7ff15c0c0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:28.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.982+0000 7ff163c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c156c30 con 0x7ff15c158560 2024-01-21T22:25:28.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.982+0000 7ff163c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c1538a0 con 0x7ff15c15e000 2024-01-21T22:25:28.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.982+0000 7ff163c00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15c154290 con 0x7ff15c0c9300 2024-01-21T22:25:28.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.983+0000 7ff161c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff15c15e000 0x7ff15c0c0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:28.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.983+0000 7ff161200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff15c0c9300 0x7ff15c1f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:28.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.983+0000 7ff161c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff15c15e000 0x7ff15c0c0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:51346/0 (socket says 172.21.15.138:51346) 2024-01-21T22:25:28.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.983+0000 7ff161c00640 1 -- 172.21.15.138:0/3639116948 learned_addr learned my addr 172.21.15.138:0/3639116948 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:28.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.985+0000 7ff15be00640 1 --2- 172.21.15.138:0/3639116948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff15c158560 0x7ff15c153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:28.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.986+0000 7ff161c00640 1 -- 172.21.15.138:0/3639116948 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff15c0c9300 msgr2=0x7ff15c1f3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:28.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.986+0000 7ff161c00640 1 --2- 172.21.15.138:0/3639116948 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff15c0c9300 0x7ff15c1f3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:28.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.986+0000 7ff161c00640 1 -- 172.21.15.138:0/3639116948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff15c158560 msgr2=0x7ff15c153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:28.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.986+0000 7ff161c00640 1 --2- 172.21.15.138:0/3639116948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff15c158560 0x7ff15c153d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:28.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.986+0000 7ff161c00640 1 -- 172.21.15.138:0/3639116948 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff144009b30 con 0x7ff15c15e000 2024-01-21T22:25:28.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.989+0000 7ff161c00640 1 --2- 172.21.15.138:0/3639116948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff15c15e000 0x7ff15c0c0e30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff15400b450 tx=0x7ff15400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:28.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.992+0000 7ff158c00640 1 -- 172.21.15.138:0/3639116948 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff154011890 con 0x7ff15c15e000 2024-01-21T22:25:28.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.992+0000 7ff158c00640 1 -- 172.21.15.138:0/3639116948 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1540119f0 con 0x7ff15c15e000 2024-01-21T22:25:28.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.992+0000 7ff158c00640 1 -- 172.21.15.138:0/3639116948 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff154011d80 con 0x7ff15c15e000 2024-01-21T22:25:28.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.992+0000 7ff163c00640 1 -- 172.21.15.138:0/3639116948 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15c156e80 con 0x7ff15c15e000 2024-01-21T22:25:28.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.992+0000 7ff163c00640 1 -- 172.21.15.138:0/3639116948 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15c0bff10 con 0x7ff15c15e000 2024-01-21T22:25:28.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:28.994+0000 7ff163c00640 1 -- 172.21.15.138:0/3639116948 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff15c153a70 con 0x7ff15c15e000 2024-01-21T22:25:29.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.246+0000 7ff158c00640 1 -- 172.21.15.138:0/3639116948 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7ff15401a400 con 0x7ff15c15e000 2024-01-21T22:25:29.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.248+0000 7ff158c00640 1 --2- 172.21.15.138:0/3639116948 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7ff13406b380 0x7ff13406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:29.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.248+0000 7ff158c00640 1 -- 172.21.15.138:0/3639116948 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff15408a330 con 0x7ff15c15e000 2024-01-21T22:25:29.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.249+0000 7ff161200640 1 --2- 172.21.15.138:0/3639116948 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7ff13406b380 0x7ff13406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:29.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.250+0000 7ff161200640 1 --2- 172.21.15.138:0/3639116948 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7ff13406b380 0x7ff13406d840 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0017f0 tx=0x7ff14c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:29.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.257+0000 7ff158c00640 1 -- 172.21.15.138:0/3639116948 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7ff1540551e0 con 0x7ff15c15e000 2024-01-21T22:25:29.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.465+0000 7ff163c00640 1 -- 172.21.15.138:0/3639116948 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff15c067ed0 con 0x7ff15c15e000 2024-01-21T22:25:29.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.470+0000 7ff158c00640 1 -- 172.21.15.138:0/3639116948 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff154058800 con 0x7ff15c15e000 2024-01-21T22:25:29.471 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 32 2024-01-21T22:25:29.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.473+0000 7ff163c00640 1 -- 172.21.15.138:0/3639116948 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7ff13406b380 msgr2=0x7ff13406d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:29.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.473+0000 7ff163c00640 1 --2- 172.21.15.138:0/3639116948 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7ff13406b380 0x7ff13406d840 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0017f0 tx=0x7ff14c009040 comp rx=0 tx=0).stop 2024-01-21T22:25:29.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.473+0000 7ff163c00640 1 -- 172.21.15.138:0/3639116948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff15c15e000 msgr2=0x7ff15c0c0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:29.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.473+0000 7ff163c00640 1 --2- 172.21.15.138:0/3639116948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff15c15e000 0x7ff15c0c0e30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff15400b450 tx=0x7ff15400b910 comp rx=0 tx=0).stop 2024-01-21T22:25:29.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.473+0000 7ff163c00640 1 -- 172.21.15.138:0/3639116948 shutdown_connections 2024-01-21T22:25:29.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.473+0000 7ff163c00640 1 --2- 172.21.15.138:0/3639116948 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7ff13406b380 0x7ff13406d840 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:29.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.473+0000 7ff163c00640 1 --2- 172.21.15.138:0/3639116948 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7ff15c15e000 0x7ff15c0c0e30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:29.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.473+0000 7ff163c00640 1 --2- 172.21.15.138:0/3639116948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff15c158560 0x7ff15c153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:29.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.473+0000 7ff163c00640 1 --2- 172.21.15.138:0/3639116948 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7ff15c0c9300 0x7ff15c1f3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:29.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.473+0000 7ff163c00640 1 -- 172.21.15.138:0/3639116948 >> 172.21.15.138:0/3639116948 conn(0x7ff15c066400 msgr2=0x7ff15c0667e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:29.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.473+0000 7ff163c00640 1 -- 172.21.15.138:0/3639116948 shutdown_connections 2024-01-21T22:25:29.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.473+0000 7ff163c00640 1 -- 172.21.15.138:0/3639116948 wait complete. 2024-01-21T22:25:29.493 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-21T22:25:29.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.697+0000 7f1578400640 1 Processor -- start 2024-01-21T22:25:29.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.697+0000 7f1578400640 1 -- start start 2024-01-21T22:25:29.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.699+0000 7f1578400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f15701583e0 0x7f1570070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:29.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.699+0000 7f1578400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15701622c0 con 0x7f15700c5b90 2024-01-21T22:25:29.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.699+0000 7f1578400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1570161460 con 0x7f1570158000 2024-01-21T22:25:29.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.699+0000 7f1578400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1570161740 con 0x7f15701583e0 2024-01-21T22:25:29.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.700+0000 7f1575a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f1570158000 0x7f15700c6450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:48558/0 (socket says 172.21.15.138:48558) 2024-01-21T22:25:29.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.700+0000 7f1575a00640 1 -- 172.21.15.138:0/3352897 learned_addr learned my addr 172.21.15.138:0/3352897 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:29.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.700+0000 7f1575000640 1 --2- 172.21.15.138:0/3352897 >> v2:172.21.15.138:3300/0 conn(0x7f15701583e0 0x7f1570070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:29.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.702+0000 7f156fe00640 1 -- 172.21.15.138:0/3352897 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1489796754 0 0) 0x7f1570161460 con 0x7f1570158000 2024-01-21T22:25:29.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.702+0000 7f156fe00640 1 -- 172.21.15.138:0/3352897 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1550003520 con 0x7f1570158000 2024-01-21T22:25:29.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.704+0000 7f156fe00640 1 -- 172.21.15.138:0/3352897 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2206142593 0 0) 0x7f15701622c0 con 0x7f15700c5b90 2024-01-21T22:25:29.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.704+0000 7f156fe00640 1 -- 172.21.15.138:0/3352897 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1570161460 con 0x7f15700c5b90 2024-01-21T22:25:29.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.705+0000 7f156fe00640 1 -- 172.21.15.138:0/3352897 <== mon.1 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2555700290 0 0) 0x7f1550003520 con 0x7f1570158000 2024-01-21T22:25:29.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.706+0000 7f156fe00640 1 -- 172.21.15.138:0/3352897 >> v2:172.21.15.138:3300/0 conn(0x7f15701583e0 msgr2=0x7f1570070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:29.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.706+0000 7f156fe00640 1 --2- 172.21.15.138:0/3352897 >> v2:172.21.15.138:3300/0 conn(0x7f15701583e0 0x7f1570070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:29.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.706+0000 7f156fe00640 1 -- 172.21.15.138:0/3352897 >> v1:172.21.15.138:6789/0 conn(0x7f15700c5b90 legacy=0x7f157015e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:29.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.706+0000 7f156fe00640 1 -- 172.21.15.138:0/3352897 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1570161a20 con 0x7f1570158000 2024-01-21T22:25:29.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.706+0000 7f156fe00640 1 -- 172.21.15.138:0/3352897 <== mon.1 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1288616872 0 0) 0x7f15600019f0 con 0x7f1570158000 2024-01-21T22:25:29.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.706+0000 7f156fe00640 1 -- 172.21.15.138:0/3352897 >> v1:172.21.15.152:6789/0 conn(0x7f1570158000 legacy=0x7f15700c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:29.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.706+0000 7f156fe00640 1 --2- 172.21.15.138:0/3352897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1550003b50 0x7f1550003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:29.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.706+0000 7f156fe00640 1 --2- 172.21.15.138:0/3352897 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f15500047c0 0x7f1550004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:29.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.706+0000 7f156fe00640 1 --2- 172.21.15.138:0/3352897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1550005180 0x7f155000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:29.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.706+0000 7f156fe00640 1 -- 172.21.15.138:0/3352897 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1570161740 con 0x7f1550003b50 2024-01-21T22:25:29.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.706+0000 7f156fe00640 1 -- 172.21.15.138:0/3352897 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155000bfa0 con 0x7f1550005180 2024-01-21T22:25:29.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.706+0000 7f156fe00640 1 -- 172.21.15.138:0/3352897 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155000c160 con 0x7f15500047c0 2024-01-21T22:25:29.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.706+0000 7f1575000640 1 --2- 172.21.15.138:0/3352897 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f15500047c0 0x7f1550004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:29.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.709+0000 7f1576400640 1 --2- 172.21.15.138:0/3352897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1550005180 0x7f155000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:29.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.710+0000 7f1575000640 1 -- 172.21.15.138:0/3352897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1550005180 msgr2=0x7f155000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:29.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.710+0000 7f1575000640 1 --2- 172.21.15.138:0/3352897 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1550005180 0x7f155000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:29.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.710+0000 7f1575000640 1 -- 172.21.15.138:0/3352897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1550003b50 msgr2=0x7f1550003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:29.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.710+0000 7f1575000640 1 --2- 172.21.15.138:0/3352897 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1550003b50 0x7f1550003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:29.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.710+0000 7f1575000640 1 -- 172.21.15.138:0/3352897 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1550003520 con 0x7f15500047c0 2024-01-21T22:25:29.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.710+0000 7f1576400640 1 -- 172.21.15.138:0/3352897 reap_dead start 2024-01-21T22:25:29.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.712+0000 7f1575000640 1 --2- 172.21.15.138:0/3352897 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f15500047c0 0x7f1550004c40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1558009ac0 tx=0x7f15500057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce89a28ba1f9c32b server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:29.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.716+0000 7f156fe00640 1 -- 172.21.15.138:0/3352897 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155802c6d0 con 0x7f15500047c0 2024-01-21T22:25:29.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.716+0000 7f156fe00640 1 -- 172.21.15.138:0/3352897 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155802c830 con 0x7f15500047c0 2024-01-21T22:25:29.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.716+0000 7f156fe00640 1 -- 172.21.15.138:0/3352897 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155802cbc0 con 0x7f15500047c0 2024-01-21T22:25:29.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.716+0000 7f1578400640 1 -- 172.21.15.138:0/3352897 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f15500047c0 msgr2=0x7f1550004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:29.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.716+0000 7f1578400640 1 --2- 172.21.15.138:0/3352897 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f15500047c0 0x7f1550004c40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1558009ac0 tx=0x7f15500057f0 comp rx=0 tx=0).stop 2024-01-21T22:25:29.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.716+0000 7f1578400640 1 -- 172.21.15.138:0/3352897 shutdown_connections 2024-01-21T22:25:29.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.717+0000 7f1578400640 1 --2- 172.21.15.138:0/3352897 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f15500047c0 0x7f1550004c40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:29.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.717+0000 7f1578400640 1 -- 172.21.15.138:0/3352897 >> 172.21.15.138:0/3352897 conn(0x7f1570072a20 msgr2=0x7f1570066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:29.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.717+0000 7f1578400640 1 -- 172.21.15.138:0/3352897 shutdown_connections 2024-01-21T22:25:29.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.717+0000 7f1578400640 1 -- 172.21.15.138:0/3352897 wait complete. 2024-01-21T22:25:29.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.719+0000 7f1578400640 1 Processor -- start 2024-01-21T22:25:29.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.719+0000 7f1578400640 1 -- start start 2024-01-21T22:25:29.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.720+0000 7f1578400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f15500047c0 0x7f15700c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:29.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.720+0000 7f1578400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f157015dc40 0x7f15701f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:29.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.720+0000 7f1578400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15700ca7c0 0x7f157014d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:29.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.720+0000 7f1578400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15700ca3a0 con 0x7f15700ca7c0 2024-01-21T22:25:29.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.720+0000 7f1578400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15700c91f0 con 0x7f157015dc40 2024-01-21T22:25:29.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.720+0000 7f1578400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1570160750 con 0x7f15500047c0 2024-01-21T22:25:29.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.721+0000 7f1575a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f15500047c0 0x7f15700c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:29.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.721+0000 7f1576400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15700ca7c0 0x7f157014d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:29.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.721+0000 7f1575000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f157015dc40 0x7f15701f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:29.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.721+0000 7f1575a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f15500047c0 0x7f15700c2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:40758/0 (socket says 172.21.15.138:40758) 2024-01-21T22:25:29.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.721+0000 7f1575a00640 1 -- 172.21.15.138:0/458007214 learned_addr learned my addr 172.21.15.138:0/458007214 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:29.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.724+0000 7f1575a00640 1 -- 172.21.15.138:0/458007214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f157015dc40 msgr2=0x7f15701f71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:29.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.724+0000 7f1575a00640 1 --2- 172.21.15.138:0/458007214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f157015dc40 0x7f15701f71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:29.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.724+0000 7f1575a00640 1 -- 172.21.15.138:0/458007214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15700ca7c0 msgr2=0x7f157014d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:29.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.724+0000 7f1575a00640 1 --2- 172.21.15.138:0/458007214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15700ca7c0 0x7f157014d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:29.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.724+0000 7f1575a00640 1 -- 172.21.15.138:0/458007214 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15701609f0 con 0x7f15500047c0 2024-01-21T22:25:29.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.727+0000 7f1575a00640 1 --2- 172.21.15.138:0/458007214 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f15500047c0 0x7f15700c2e50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f156000cad0 tx=0x7f156000ced0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:29.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.730+0000 7f156d600640 1 -- 172.21.15.138:0/458007214 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156000a700 con 0x7f15500047c0 2024-01-21T22:25:29.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.730+0000 7f156d600640 1 -- 172.21.15.138:0/458007214 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156000a860 con 0x7f15500047c0 2024-01-21T22:25:29.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.730+0000 7f156d600640 1 -- 172.21.15.138:0/458007214 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156000abf0 con 0x7f15500047c0 2024-01-21T22:25:29.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.730+0000 7f1578400640 1 -- 172.21.15.138:0/458007214 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1570160cd0 con 0x7f15500047c0 2024-01-21T22:25:29.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.731+0000 7f1578400640 1 -- 172.21.15.138:0/458007214 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15700c9fc0 con 0x7f15500047c0 2024-01-21T22:25:29.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.732+0000 7f1578400640 1 -- 172.21.15.138:0/458007214 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15700ca180 con 0x7f15500047c0 2024-01-21T22:25:29.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.752+0000 7f156d600640 1 -- 172.21.15.138:0/458007214 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f156000ad50 con 0x7f15500047c0 2024-01-21T22:25:29.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.753+0000 7f156d600640 1 --2- 172.21.15.138:0/458007214 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f154806b380 0x7f154806d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:29.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.753+0000 7f1575000640 1 --2- 172.21.15.138:0/458007214 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f154806b380 0x7f154806d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:29.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.754+0000 7f156d600640 1 -- 172.21.15.138:0/458007214 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1560007e20 con 0x7f15500047c0 2024-01-21T22:25:29.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.754+0000 7f1575000640 1 --2- 172.21.15.138:0/458007214 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f154806b380 0x7f154806d840 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1558036000 tx=0x7f1558030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:29.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:29.986+0000 7f156d600640 1 -- 172.21.15.138:0/458007214 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f15600b50d0 con 0x7f15500047c0 2024-01-21T22:25:30.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.197+0000 7f1578400640 1 -- 172.21.15.138:0/458007214 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f15701611f0 con 0x7f15500047c0 2024-01-21T22:25:30.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.266+0000 7f156d600640 1 -- 172.21.15.138:0/458007214 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f15701611f0 con 0x7f15500047c0 2024-01-21T22:25:30.267 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 64 2024-01-21T22:25:30.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.269+0000 7f1578400640 1 -- 172.21.15.138:0/458007214 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f154806b380 msgr2=0x7f154806d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:30.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.269+0000 7f1578400640 1 --2- 172.21.15.138:0/458007214 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f154806b380 0x7f154806d840 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1558036000 tx=0x7f1558030000 comp rx=0 tx=0).stop 2024-01-21T22:25:30.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.269+0000 7f1578400640 1 -- 172.21.15.138:0/458007214 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f15500047c0 msgr2=0x7f15700c2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:30.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.269+0000 7f1578400640 1 --2- 172.21.15.138:0/458007214 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f15500047c0 0x7f15700c2e50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f156000cad0 tx=0x7f156000ced0 comp rx=0 tx=0).stop 2024-01-21T22:25:30.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.270+0000 7f1578400640 1 -- 172.21.15.138:0/458007214 shutdown_connections 2024-01-21T22:25:30.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.270+0000 7f1578400640 1 --2- 172.21.15.138:0/458007214 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f154806b380 0x7f154806d840 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:30.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.270+0000 7f1578400640 1 --2- 172.21.15.138:0/458007214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15700ca7c0 0x7f157014d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:30.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.270+0000 7f1578400640 1 --2- 172.21.15.138:0/458007214 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f157015dc40 0x7f15701f71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:30.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.270+0000 7f1578400640 1 --2- 172.21.15.138:0/458007214 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f15500047c0 0x7f15700c2e50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:30.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.270+0000 7f1578400640 1 -- 172.21.15.138:0/458007214 >> 172.21.15.138:0/458007214 conn(0x7f1570072a20 msgr2=0x7f1570071270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:30.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.270+0000 7f1578400640 1 -- 172.21.15.138:0/458007214 shutdown_connections 2024-01-21T22:25:30.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.270+0000 7f1578400640 1 -- 172.21.15.138:0/458007214 wait complete. 2024-01-21T22:25:30.291 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-21T22:25:30.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.498+0000 7f096b400640 1 Processor -- start 2024-01-21T22:25:30.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.498+0000 7f096b400640 1 -- start start 2024-01-21T22:25:30.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.500+0000 7f096b400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f09640716e0 0x7f09640c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:30.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.500+0000 7f096b400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f09640c7520 0x7f096406ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:30.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.500+0000 7f096b400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0964157740 con 0x7f0964150a00 2024-01-21T22:25:30.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.500+0000 7f096b400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09640c4cc0 con 0x7f09640c7520 2024-01-21T22:25:30.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.500+0000 7f096b400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096406a920 con 0x7f09640716e0 2024-01-21T22:25:30.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.500+0000 7f0968a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f0964150a00 0x7f09640c25b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:60778/0 (socket says 172.21.15.138:60778) 2024-01-21T22:25:30.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.500+0000 7f0963e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f09640716e0 0x7f09640c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:30.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.500+0000 7f0968a00640 1 -- 172.21.15.138:0/3724439464 learned_addr learned my addr 172.21.15.138:0/3724439464 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:30.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.500+0000 7f0969400640 1 --2- 172.21.15.138:0/3724439464 >> v2:172.21.15.152:3300/0 conn(0x7f09640c7520 0x7f096406ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:30.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.503+0000 7f0963400640 1 -- 172.21.15.138:0/3724439464 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1620427142 0 0) 0x7f0964157740 con 0x7f0964150a00 2024-01-21T22:25:30.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.503+0000 7f0963400640 1 -- 172.21.15.138:0/3724439464 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0944005730 con 0x7f0964150a00 2024-01-21T22:25:30.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.503+0000 7f0963e00640 1 -- 172.21.15.138:0/3724439464 >> v2:172.21.15.152:3300/0 conn(0x7f09640c7520 msgr2=0x7f096406ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:30.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.503+0000 7f0963e00640 1 --2- 172.21.15.138:0/3724439464 >> v2:172.21.15.152:3300/0 conn(0x7f09640c7520 0x7f096406ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:30.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.503+0000 7f0963e00640 1 -- 172.21.15.138:0/3724439464 >> v1:172.21.15.138:6789/0 conn(0x7f0964150a00 legacy=0x7f09640c25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:30.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.504+0000 7f0963e00640 1 -- 172.21.15.138:0/3724439464 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0964156bc0 con 0x7f09640716e0 2024-01-21T22:25:30.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.506+0000 7f0963e00640 1 --2- 172.21.15.138:0/3724439464 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f09640716e0 0x7f09640c6fe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f094c0097a0 tx=0x7f094c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c3771c0e73f0029 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:30.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.510+0000 7f0963400640 1 -- 172.21.15.138:0/3724439464 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c02e070 con 0x7f09640716e0 2024-01-21T22:25:30.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.510+0000 7f0963400640 1 -- 172.21.15.138:0/3724439464 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c0315a0 con 0x7f09640716e0 2024-01-21T22:25:30.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.510+0000 7f0963400640 1 -- 172.21.15.138:0/3724439464 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c031930 con 0x7f09640716e0 2024-01-21T22:25:30.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.511+0000 7f096b400640 1 -- 172.21.15.138:0/3724439464 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f09640716e0 msgr2=0x7f09640c6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:30.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.511+0000 7f096b400640 1 --2- 172.21.15.138:0/3724439464 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f09640716e0 0x7f09640c6fe0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f094c0097a0 tx=0x7f094c02b840 comp rx=0 tx=0).stop 2024-01-21T22:25:30.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.511+0000 7f096b400640 1 -- 172.21.15.138:0/3724439464 shutdown_connections 2024-01-21T22:25:30.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.511+0000 7f096b400640 1 --2- 172.21.15.138:0/3724439464 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f09640716e0 0x7f09640c6fe0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:30.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.511+0000 7f096b400640 1 --2- 172.21.15.138:0/3724439464 >> v2:172.21.15.152:3300/0 conn(0x7f09640c7520 0x7f096406ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:30.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.511+0000 7f096b400640 1 -- 172.21.15.138:0/3724439464 >> 172.21.15.138:0/3724439464 conn(0x7f09640cb0b0 msgr2=0x7f09640cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:30.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.511+0000 7f096b400640 1 -- 172.21.15.138:0/3724439464 shutdown_connections 2024-01-21T22:25:30.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.511+0000 7f096b400640 1 -- 172.21.15.138:0/3724439464 wait complete. 2024-01-21T22:25:30.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.513+0000 7f096b400640 1 Processor -- start 2024-01-21T22:25:30.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.513+0000 7f096b400640 1 -- start start 2024-01-21T22:25:30.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.514+0000 7f096b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09640716e0 0x7f09641eeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:30.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.514+0000 7f096b400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f09640c7520 0x7f096413dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:30.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.514+0000 7f096b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0964150a00 0x7f0964143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:30.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.514+0000 7f096b400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09640c9510 con 0x7f0964150a00 2024-01-21T22:25:30.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.514+0000 7f096b400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09640c9170 con 0x7f09640716e0 2024-01-21T22:25:30.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.514+0000 7f096b400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0964150220 con 0x7f09640c7520 2024-01-21T22:25:30.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.514+0000 7f0963e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f09640c7520 0x7f096413dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:30.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.514+0000 7f0969400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0964150a00 0x7f0964143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:30.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.515+0000 7f0968a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09640716e0 0x7f09641eeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:30.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.515+0000 7f0963e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f09640c7520 0x7f096413dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:40780/0 (socket says 172.21.15.138:40780) 2024-01-21T22:25:30.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.515+0000 7f0968a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09640716e0 0x7f09641eeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:51386/0 (socket says 172.21.15.138:51386) 2024-01-21T22:25:30.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.515+0000 7f0963e00640 1 -- 172.21.15.138:0/2510828372 learned_addr learned my addr 172.21.15.138:0/2510828372 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:30.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.518+0000 7f0969400640 1 -- 172.21.15.138:0/2510828372 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f09640c7520 msgr2=0x7f096413dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:30.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.518+0000 7f0969400640 1 --2- 172.21.15.138:0/2510828372 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f09640c7520 0x7f096413dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:30.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.518+0000 7f0969400640 1 -- 172.21.15.138:0/2510828372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09640716e0 msgr2=0x7f09641eeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:30.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.518+0000 7f0969400640 1 --2- 172.21.15.138:0/2510828372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09640716e0 0x7f09641eeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:30.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.518+0000 7f0969400640 1 -- 172.21.15.138:0/2510828372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094c009400 con 0x7f0964150a00 2024-01-21T22:25:30.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.521+0000 7f0969400640 1 --2- 172.21.15.138:0/2510828372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0964150a00 0x7f0964143120 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f095c00efc0 tx=0x7f095c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:30.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.525+0000 7f0960c00640 1 -- 172.21.15.138:0/2510828372 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c009c00 con 0x7f0964150a00 2024-01-21T22:25:30.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.525+0000 7f0960c00640 1 -- 172.21.15.138:0/2510828372 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095c00f040 con 0x7f0964150a00 2024-01-21T22:25:30.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.525+0000 7f0960c00640 1 -- 172.21.15.138:0/2510828372 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c0146c0 con 0x7f0964150a00 2024-01-21T22:25:30.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.525+0000 7f096b400640 1 -- 172.21.15.138:0/2510828372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09641ef510 con 0x7f0964150a00 2024-01-21T22:25:30.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.526+0000 7f096b400640 1 -- 172.21.15.138:0/2510828372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09641efad0 con 0x7f0964150a00 2024-01-21T22:25:30.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.527+0000 7f096b400640 1 -- 172.21.15.138:0/2510828372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09640c9650 con 0x7f0964150a00 2024-01-21T22:25:30.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.547+0000 7f0960c00640 1 -- 172.21.15.138:0/2510828372 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f095c014820 con 0x7f0964150a00 2024-01-21T22:25:30.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.549+0000 7f0960c00640 1 --2- 172.21.15.138:0/2510828372 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f093c06b380 0x7f093c06d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:30.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.549+0000 7f0968a00640 1 --2- 172.21.15.138:0/2510828372 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f093c06b380 0x7f093c06d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:30.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.550+0000 7f0968a00640 1 --2- 172.21.15.138:0/2510828372 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f093c06b380 0x7f093c06d840 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0954001200 tx=0x7f0954000bc0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:30.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.839+0000 7f0960c00640 1 -- 172.21.15.138:0/2510828372 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f095c059760 con 0x7f0964150a00 2024-01-21T22:25:30.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:30.840+0000 7f0960c00640 1 -- 172.21.15.138:0/2510828372 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f095c055aa0 con 0x7f0964150a00 2024-01-21T22:25:31.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.087+0000 7f096b400640 1 -- 172.21.15.138:0/2510828372 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09640c9650 con 0x7f0964150a00 2024-01-21T22:25:31.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.232+0000 7f0960c00640 1 -- 172.21.15.138:0/2510828372 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+6530 (secure 0 0 0) 0x7f09640c9650 con 0x7f0964150a00 2024-01-21T22:25:31.234 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:25:31.234 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/3112363661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":3112363661},{"type":"v1","addr":"172.21.15.152:6837","nonce":3112363661}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4223,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/3222259940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":3222259940},{"type":"v1","addr":"172.21.15.152:6839","nonce":3222259940}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/909784319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":909784319},{"type":"v1","addr":"172.21.15.138:6833","nonce":909784319}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4229,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3318550662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3318550662},{"type":"v1","addr":"172.21.15.138:6835","nonce":3318550662}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2986531362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2986531362},{"type":"v1","addr":"172.21.15.138:6837","nonce":2986531362}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-21T22:25:04.967250+0000","modified":"2024-01-21T22:25:12.598970+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":5,"state":"up:active","state_seq":5,"addr":"172.21.15.152:6835/2663179600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2663179600},{"type":"v1","addr":"172.21.15.152:6835","nonce":2663179600}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-21T22:25:31.234 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 10 2024-01-21T22:25:31.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.235+0000 7f096b400640 1 -- 172.21.15.138:0/2510828372 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f093c06b380 msgr2=0x7f093c06d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:31.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.235+0000 7f096b400640 1 --2- 172.21.15.138:0/2510828372 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f093c06b380 0x7f093c06d840 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0954001200 tx=0x7f0954000bc0 comp rx=0 tx=0).stop 2024-01-21T22:25:31.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.236+0000 7f096b400640 1 -- 172.21.15.138:0/2510828372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0964150a00 msgr2=0x7f0964143120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:31.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.236+0000 7f096b400640 1 --2- 172.21.15.138:0/2510828372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0964150a00 0x7f0964143120 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f095c00efc0 tx=0x7f095c00c490 comp rx=0 tx=0).stop 2024-01-21T22:25:31.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.236+0000 7f096b400640 1 -- 172.21.15.138:0/2510828372 shutdown_connections 2024-01-21T22:25:31.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.236+0000 7f096b400640 1 --2- 172.21.15.138:0/2510828372 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f093c06b380 0x7f093c06d840 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:31.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.236+0000 7f096b400640 1 --2- 172.21.15.138:0/2510828372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0964150a00 0x7f0964143120 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:31.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.236+0000 7f096b400640 1 --2- 172.21.15.138:0/2510828372 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f09640c7520 0x7f096413dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:31.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.236+0000 7f096b400640 1 --2- 172.21.15.138:0/2510828372 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f09640716e0 0x7f09641eeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:31.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.236+0000 7f096b400640 1 -- 172.21.15.138:0/2510828372 >> 172.21.15.138:0/2510828372 conn(0x7f09640cb0b0 msgr2=0x7f09640c64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:31.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.236+0000 7f096b400640 1 -- 172.21.15.138:0/2510828372 shutdown_connections 2024-01-21T22:25:31.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.236+0000 7f096b400640 1 -- 172.21.15.138:0/2510828372 wait complete. 2024-01-21T22:25:31.262 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-21T22:25:31.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.462+0000 7fb336a00640 1 Processor -- start 2024-01-21T22:25:31.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.462+0000 7fb336a00640 1 -- start start 2024-01-21T22:25:31.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.463+0000 7fb336a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb3300716e0 0x7fb3300c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:31.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.463+0000 7fb336a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb330157740 con 0x7fb330150a00 2024-01-21T22:25:31.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.463+0000 7fb336a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb330156bc0 con 0x7fb3300c7520 2024-01-21T22:25:31.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.463+0000 7fb336a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3300c4cc0 con 0x7fb3300716e0 2024-01-21T22:25:31.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.464+0000 7fb32f400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb3300716e0 0x7fb3300c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:31.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.465+0000 7fb32f400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb3300716e0 0x7fb3300c6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:51392/0 (socket says 172.21.15.138:51392) 2024-01-21T22:25:31.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.465+0000 7fb32f400640 1 -- 172.21.15.138:0/185703803 learned_addr learned my addr 172.21.15.138:0/185703803 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:31.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.467+0000 7fb32f400640 1 -- 172.21.15.138:0/185703803 >> v1:172.21.15.152:6789/0 conn(0x7fb3300c7520 legacy=0x7fb33006ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:31.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.467+0000 7fb32f400640 1 -- 172.21.15.138:0/185703803 >> v1:172.21.15.138:6789/0 conn(0x7fb330150a00 legacy=0x7fb3300c25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:31.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.467+0000 7fb32f400640 1 -- 172.21.15.138:0/185703803 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb330156ea0 con 0x7fb3300716e0 2024-01-21T22:25:31.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.473+0000 7fb32f400640 1 --2- 172.21.15.138:0/185703803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3300716e0 0x7fb3300c6fe0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb318009870 tx=0x7fb31802b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e1276edc433a24d server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:31.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.476+0000 7fb32ea00640 1 -- 172.21.15.138:0/185703803 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318030860 con 0x7fb3300716e0 2024-01-21T22:25:31.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.476+0000 7fb32ea00640 1 -- 172.21.15.138:0/185703803 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3180309c0 con 0x7fb3300716e0 2024-01-21T22:25:31.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.476+0000 7fb32ea00640 1 -- 172.21.15.138:0/185703803 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318030d50 con 0x7fb3300716e0 2024-01-21T22:25:31.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.477+0000 7fb336a00640 1 -- 172.21.15.138:0/185703803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3300716e0 msgr2=0x7fb3300c6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:31.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.477+0000 7fb336a00640 1 --2- 172.21.15.138:0/185703803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3300716e0 0x7fb3300c6fe0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb318009870 tx=0x7fb31802b9a0 comp rx=0 tx=0).stop 2024-01-21T22:25:31.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.477+0000 7fb336a00640 1 -- 172.21.15.138:0/185703803 shutdown_connections 2024-01-21T22:25:31.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.477+0000 7fb336a00640 1 --2- 172.21.15.138:0/185703803 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb3300716e0 0x7fb3300c6fe0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:31.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.477+0000 7fb336a00640 1 -- 172.21.15.138:0/185703803 >> 172.21.15.138:0/185703803 conn(0x7fb3300cb0b0 msgr2=0x7fb3300cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:31.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.477+0000 7fb336a00640 1 -- 172.21.15.138:0/185703803 shutdown_connections 2024-01-21T22:25:31.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.477+0000 7fb336a00640 1 -- 172.21.15.138:0/185703803 wait complete. 2024-01-21T22:25:31.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.479+0000 7fb336a00640 1 Processor -- start 2024-01-21T22:25:31.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.479+0000 7fb336a00640 1 -- start start 2024-01-21T22:25:31.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.480+0000 7fb336a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb3300716e0 0x7fb33013ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:31.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.480+0000 7fb336a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3300c7520 0x7fb3301533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:31.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.480+0000 7fb336a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb330150a00 0x7fb330153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:31.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.480+0000 7fb336a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33006a920 con 0x7fb3300c7520 2024-01-21T22:25:31.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.480+0000 7fb336a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3300c8fb0 con 0x7fb330150a00 2024-01-21T22:25:31.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.480+0000 7fb336a00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33013a380 con 0x7fb3300716e0 2024-01-21T22:25:31.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.481+0000 7fb32f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3300c7520 0x7fb3301533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:31.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.481+0000 7fb334a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb330150a00 0x7fb330153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:31.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.481+0000 7fb32fe00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb3300716e0 0x7fb33013ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:31.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.481+0000 7fb32f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3300c7520 0x7fb3301533f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40168/0 (socket says 172.21.15.138:40168) 2024-01-21T22:25:31.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.481+0000 7fb32f400640 1 -- 172.21.15.138:0/2979520475 learned_addr learned my addr 172.21.15.138:0/2979520475 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:31.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.486+0000 7fb334a00640 1 -- 172.21.15.138:0/2979520475 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb3300716e0 msgr2=0x7fb33013ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:31.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.486+0000 7fb334a00640 1 --2- 172.21.15.138:0/2979520475 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb3300716e0 0x7fb33013ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:31.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.486+0000 7fb334a00640 1 -- 172.21.15.138:0/2979520475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3300c7520 msgr2=0x7fb3301533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:31.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.486+0000 7fb334a00640 1 --2- 172.21.15.138:0/2979520475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3300c7520 0x7fb3301533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:31.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.486+0000 7fb334a00640 1 -- 172.21.15.138:0/2979520475 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3180094d0 con 0x7fb330150a00 2024-01-21T22:25:31.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.489+0000 7fb334a00640 1 --2- 172.21.15.138:0/2979520475 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb330150a00 0x7fb330153930 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb32800c690 tx=0x7fb32800cb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:31.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.492+0000 7fb327e00640 1 -- 172.21.15.138:0/2979520475 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328003b70 con 0x7fb330150a00 2024-01-21T22:25:31.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.492+0000 7fb327e00640 1 -- 172.21.15.138:0/2979520475 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb328003cd0 con 0x7fb330150a00 2024-01-21T22:25:31.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.492+0000 7fb327e00640 1 -- 172.21.15.138:0/2979520475 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328005310 con 0x7fb330150a00 2024-01-21T22:25:31.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.492+0000 7fb336a00640 1 -- 172.21.15.138:0/2979520475 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb330139da0 con 0x7fb330150a00 2024-01-21T22:25:31.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.493+0000 7fb336a00640 1 -- 172.21.15.138:0/2979520475 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb330139ac0 con 0x7fb330150a00 2024-01-21T22:25:31.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.494+0000 7fb336a00640 1 -- 172.21.15.138:0/2979520475 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33004eaa0 con 0x7fb330150a00 2024-01-21T22:25:31.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.513+0000 7fb327e00640 1 -- 172.21.15.138:0/2979520475 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fb328005470 con 0x7fb330150a00 2024-01-21T22:25:31.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.514+0000 7fb327e00640 1 --2- 172.21.15.138:0/2979520475 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fb30806b380 0x7fb30806d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:31.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.514+0000 7fb327e00640 1 -- 172.21.15.138:0/2979520475 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb328058d00 con 0x7fb330150a00 2024-01-21T22:25:31.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.514+0000 7fb32fe00640 1 --2- 172.21.15.138:0/2979520475 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fb30806b380 0x7fb30806d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:31.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.515+0000 7fb32fe00640 1 --2- 172.21.15.138:0/2979520475 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fb30806b380 0x7fb30806d840 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb3200016e0 tx=0x7fb320009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:31.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.747+0000 7fb327e00640 1 -- 172.21.15.138:0/2979520475 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fb328055040 con 0x7fb330150a00 2024-01-21T22:25:31.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.945+0000 7fb336a00640 1 -- 172.21.15.138:0/2979520475 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3300c5580 con 0x7fb330150a00 2024-01-21T22:25:31.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.981+0000 7fb327e00640 1 -- 172.21.15.138:0/2979520475 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16522 (secure 0 0 0) 0x7fb328058660 con 0x7fb330150a00 2024-01-21T22:25:31.983 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:25:31.983 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":23,"fsid":"d488d446-149f-48d0-89ca-b2830ae72846","created":"2024-01-21T22:24:28.581932+0000","modified":"2024-01-21T22:25:06.475790+0000","last_up_change":"2024-01-21T22:24:47.145799+0000","last_in_change":"2024-01-21T22:24:37.672807+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-21T22:24:49.586178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-21T22:25:02.044454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-21T22:25:03.361765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d812b2d6-c8a3-4015-bfdb-5766b25eae5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6825","nonce":2849203277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6827","nonce":2849203277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6831","nonce":2849203277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6829","nonce":2849203277}]},"public_addr":"172.21.15.138:6825/2849203277","cluster_addr":"172.21.15.138:6827/2849203277","heartbeat_back_addr":"172.21.15.138:6831/2849203277","heartbeat_front_addr":"172.21.15.138:6829/2849203277","state":["exists","up"]},{"osd":1,"uuid":"d8d875df-c840-46cc-b6c0-a847f9c93999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6817","nonce":174521731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6819","nonce":174521731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6823","nonce":174521731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6821","nonce":174521731}]},"public_addr":"172.21.15.138:6817/174521731","cluster_addr":"172.21.15.138:6819/174521731","heartbeat_back_addr":"172.21.15.138:6823/174521731","heartbeat_front_addr":"172.21.15.138:6821/174521731","state":["exists","up"]},{"osd":2,"uuid":"4bd94373-76ca-47a6-870b-16d96d7c9e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6801","nonce":1365838748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6803","nonce":1365838748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6809","nonce":1365838748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6805","nonce":1365838748}]},"public_addr":"172.21.15.138:6801/1365838748","cluster_addr":"172.21.15.138:6803/1365838748","heartbeat_back_addr":"172.21.15.138:6809/1365838748","heartbeat_front_addr":"172.21.15.138:6805/1365838748","state":["exists","up"]},{"osd":3,"uuid":"07267087-d4d2-4e88-ab9b-bb87f6dd061a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6808","nonce":1841873885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6811","nonce":1841873885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6815","nonce":1841873885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6813","nonce":1841873885}]},"public_addr":"172.21.15.138:6808/1841873885","cluster_addr":"172.21.15.138:6811/1841873885","heartbeat_back_addr":"172.21.15.138:6815/1841873885","heartbeat_front_addr":"172.21.15.138:6813/1841873885","state":["exists","up"]},{"osd":4,"uuid":"b47ef758-d783-4168-a9a5-11db3de4ca12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6827","nonce":3960812298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6829","nonce":3960812298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6833","nonce":3960812298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6831","nonce":3960812298}]},"public_addr":"172.21.15.152:6827/3960812298","cluster_addr":"172.21.15.152:6829/3960812298","heartbeat_back_addr":"172.21.15.152:6833/3960812298","heartbeat_front_addr":"172.21.15.152:6831/3960812298","state":["exists","up"]},{"osd":5,"uuid":"0ea5d43e-61d8-4823-815f-9e42c62fae97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6819","nonce":4095234314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6821","nonce":4095234314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6825","nonce":4095234314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6823","nonce":4095234314}]},"public_addr":"172.21.15.152:6819/4095234314","cluster_addr":"172.21.15.152:6821/4095234314","heartbeat_back_addr":"172.21.15.152:6825/4095234314","heartbeat_front_addr":"172.21.15.152:6823/4095234314","state":["exists","up"]},{"osd":6,"uuid":"59ffac36-e2e0-4693-ac82-ad5eccf6224d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6803","nonce":223848127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6805","nonce":223848127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6809","nonce":223848127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6807","nonce":223848127}]},"public_addr":"172.21.15.152:6803/223848127","cluster_addr":"172.21.15.152:6805/223848127","heartbeat_back_addr":"172.21.15.152:6809/223848127","heartbeat_front_addr":"172.21.15.152:6807/223848127","state":["exists","up"]},{"osd":7,"uuid":"4f892c7d-32a5-4960-bf88-4ed7c863cd0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6811","nonce":52509095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6813","nonce":52509095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6817","nonce":52509095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6815","nonce":52509095}]},"public_addr":"172.21.15.152:6811/52509095","cluster_addr":"172.21.15.152:6813/52509095","heartbeat_back_addr":"172.21.15.152:6817/52509095","heartbeat_front_addr":"172.21.15.152:6815/52509095","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.434460+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.374709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.613134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.298148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.242144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.198102+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.093385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.197180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-21T22:25:31.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.984+0000 7fb336a00640 1 -- 172.21.15.138:0/2979520475 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fb30806b380 msgr2=0x7fb30806d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:31.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.984+0000 7fb336a00640 1 --2- 172.21.15.138:0/2979520475 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fb30806b380 0x7fb30806d840 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb3200016e0 tx=0x7fb320009040 comp rx=0 tx=0).stop 2024-01-21T22:25:31.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.984+0000 7fb336a00640 1 -- 172.21.15.138:0/2979520475 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb330150a00 msgr2=0x7fb330153930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:31.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.984+0000 7fb336a00640 1 --2- 172.21.15.138:0/2979520475 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb330150a00 0x7fb330153930 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb32800c690 tx=0x7fb32800cb50 comp rx=0 tx=0).stop 2024-01-21T22:25:31.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.984+0000 7fb336a00640 1 -- 172.21.15.138:0/2979520475 shutdown_connections 2024-01-21T22:25:31.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.984+0000 7fb336a00640 1 --2- 172.21.15.138:0/2979520475 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fb30806b380 0x7fb30806d840 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:31.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.984+0000 7fb336a00640 1 --2- 172.21.15.138:0/2979520475 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb330150a00 0x7fb330153930 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:31.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.984+0000 7fb336a00640 1 --2- 172.21.15.138:0/2979520475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3300c7520 0x7fb3301533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:31.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.984+0000 7fb336a00640 1 --2- 172.21.15.138:0/2979520475 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb3300716e0 0x7fb33013ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:31.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.984+0000 7fb336a00640 1 -- 172.21.15.138:0/2979520475 >> 172.21.15.138:0/2979520475 conn(0x7fb3300cb0b0 msgr2=0x7fb3300c24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:31.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.984+0000 7fb336a00640 1 -- 172.21.15.138:0/2979520475 shutdown_connections 2024-01-21T22:25:31.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:31.984+0000 7fb336a00640 1 -- 172.21.15.138:0/2979520475 wait complete. 2024-01-21T22:25:32.004 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-21T22:24:49.586178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-21T22:25:02.044454+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-21T22:25:03.361765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-21T22:25:32.005 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-21T22:25:32.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.218+0000 7f5bce600640 1 Processor -- start 2024-01-21T22:25:32.220 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.219+0000 7f5bce600640 1 -- start start 2024-01-21T22:25:32.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.220+0000 7f5bce600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5bc80c6c90 0x7f5bc815a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:32.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.220+0000 7f5bce600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5bc80c6620 0x7f5bc80c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:32.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.220+0000 7f5bce600640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc815aad0 con 0x7f5bc8153dd0 2024-01-21T22:25:32.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.220+0000 7f5bce600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc80c8db0 con 0x7f5bc80c6620 2024-01-21T22:25:32.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.220+0000 7f5bce600640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc80c9c20 con 0x7f5bc80c6c90 2024-01-21T22:25:32.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.221+0000 7f5bc7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5bc80c6c90 0x7f5bc815a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:32.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.221+0000 7f5bcdc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5bc80c6620 0x7f5bc80c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:32.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.221+0000 7f5bc7e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5bc80c6c90 0x7f5bc815a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:51406/0 (socket says 172.21.15.138:51406) 2024-01-21T22:25:32.222 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.221+0000 7f5bc7e00640 1 -- 172.21.15.138:0/3435960788 learned_addr learned my addr 172.21.15.138:0/3435960788 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:32.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.224+0000 7f5bcdc00640 1 -- 172.21.15.138:0/3435960788 >> v2:172.21.15.152:3300/0 conn(0x7f5bc80c6c90 msgr2=0x7f5bc815a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:32.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.224+0000 7f5bcdc00640 1 --2- 172.21.15.138:0/3435960788 >> v2:172.21.15.152:3300/0 conn(0x7f5bc80c6c90 0x7f5bc815a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:32.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.224+0000 7f5bcdc00640 1 -- 172.21.15.138:0/3435960788 >> v1:172.21.15.152:6789/0 conn(0x7f5bc8153dd0 legacy=0x7f5bc80cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:32.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.224+0000 7f5bcdc00640 1 -- 172.21.15.138:0/3435960788 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc80c9ec0 con 0x7f5bc80c6620 2024-01-21T22:25:32.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.227+0000 7f5bcdc00640 1 --2- 172.21.15.138:0/3435960788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bc80c6620 0x7f5bc80c74a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5bc806bcc0 tx=0x7f5bc002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15dd29e17e5cd7c3 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:32.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.232+0000 7f5bc7400640 1 -- 172.21.15.138:0/3435960788 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc002cd00 con 0x7f5bc80c6620 2024-01-21T22:25:32.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.232+0000 7f5bc7400640 1 -- 172.21.15.138:0/3435960788 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc002ce60 con 0x7f5bc80c6620 2024-01-21T22:25:32.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.232+0000 7f5bc7400640 1 -- 172.21.15.138:0/3435960788 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0034660 con 0x7f5bc80c6620 2024-01-21T22:25:32.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.233+0000 7f5bce600640 1 -- 172.21.15.138:0/3435960788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bc80c6620 msgr2=0x7f5bc80c74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:32.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.233+0000 7f5bce600640 1 --2- 172.21.15.138:0/3435960788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bc80c6620 0x7f5bc80c74a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5bc806bcc0 tx=0x7f5bc002bf50 comp rx=0 tx=0).stop 2024-01-21T22:25:32.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.234+0000 7f5bce600640 1 -- 172.21.15.138:0/3435960788 shutdown_connections 2024-01-21T22:25:32.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.234+0000 7f5bce600640 1 --2- 172.21.15.138:0/3435960788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bc80c6620 0x7f5bc80c74a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:32.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.234+0000 7f5bce600640 1 --2- 172.21.15.138:0/3435960788 >> v2:172.21.15.152:3300/0 conn(0x7f5bc80c6c90 0x7f5bc815a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:32.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.234+0000 7f5bce600640 1 -- 172.21.15.138:0/3435960788 >> 172.21.15.138:0/3435960788 conn(0x7f5bc80cf100 msgr2=0x7f5bc80cf510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:32.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.234+0000 7f5bce600640 1 -- 172.21.15.138:0/3435960788 shutdown_connections 2024-01-21T22:25:32.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.234+0000 7f5bce600640 1 -- 172.21.15.138:0/3435960788 wait complete. 2024-01-21T22:25:32.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.237+0000 7f5bce600640 1 Processor -- start 2024-01-21T22:25:32.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.237+0000 7f5bce600640 1 -- start start 2024-01-21T22:25:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.239+0000 7f5bce600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bc80c6620 0x7f5bc8154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.239+0000 7f5bce600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bc80c6c90 0x7f5bc814b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.239+0000 7f5bce600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5bc8153dd0 0x7f5bc80c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.239+0000 7f5bce600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc81549c0 con 0x7f5bc80c6c90 2024-01-21T22:25:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.239+0000 7f5bce600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc814bd90 con 0x7f5bc80c6620 2024-01-21T22:25:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.239+0000 7f5bce600640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc8154c00 con 0x7f5bc8153dd0 2024-01-21T22:25:32.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.239+0000 7f5bcd200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bc80c6620 0x7f5bc8154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:32.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.239+0000 7f5bcdc00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5bc8153dd0 0x7f5bc80c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:32.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.240+0000 7f5bc7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bc80c6c90 0x7f5bc814b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:32.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.240+0000 7f5bcd200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bc80c6620 0x7f5bc8154480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:51414/0 (socket says 172.21.15.138:51414) 2024-01-21T22:25:32.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.240+0000 7f5bcd200640 1 -- 172.21.15.138:0/1573264401 learned_addr learned my addr 172.21.15.138:0/1573264401 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:32.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.243+0000 7f5bcd200640 1 -- 172.21.15.138:0/1573264401 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5bc8153dd0 msgr2=0x7f5bc80c47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:32.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.243+0000 7f5bcd200640 1 --2- 172.21.15.138:0/1573264401 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5bc8153dd0 0x7f5bc80c47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:32.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.243+0000 7f5bcd200640 1 -- 172.21.15.138:0/1573264401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bc80c6c90 msgr2=0x7f5bc814b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:32.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.243+0000 7f5bcd200640 1 --2- 172.21.15.138:0/1573264401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bc80c6c90 0x7f5bc814b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:32.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.243+0000 7f5bcd200640 1 -- 172.21.15.138:0/1573264401 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc0009a70 con 0x7f5bc80c6620 2024-01-21T22:25:32.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.246+0000 7f5bcd200640 1 --2- 172.21.15.138:0/1573264401 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bc80c6620 0x7f5bc8154480 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800c3c0 tx=0x7f5bb800c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:32.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.249+0000 7f5bc4c00640 1 -- 172.21.15.138:0/1573264401 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb80052d0 con 0x7f5bc80c6620 2024-01-21T22:25:32.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.249+0000 7f5bc4c00640 1 -- 172.21.15.138:0/1573264401 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb8005450 con 0x7f5bc80c6620 2024-01-21T22:25:32.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.249+0000 7f5bc4c00640 1 -- 172.21.15.138:0/1573264401 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8007260 con 0x7f5bc80c6620 2024-01-21T22:25:32.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.250+0000 7f5bce600640 1 -- 172.21.15.138:0/1573264401 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc8154ec0 con 0x7f5bc80c6620 2024-01-21T22:25:32.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.250+0000 7f5bce600640 1 -- 172.21.15.138:0/1573264401 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc8156970 con 0x7f5bc80c6620 2024-01-21T22:25:32.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.251+0000 7f5bce600640 1 -- 172.21.15.138:0/1573264401 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc8155690 con 0x7f5bc80c6620 2024-01-21T22:25:32.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.270+0000 7f5bc4c00640 1 -- 172.21.15.138:0/1573264401 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f5bb80073c0 con 0x7f5bc80c6620 2024-01-21T22:25:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.271+0000 7f5bc4c00640 1 --2- 172.21.15.138:0/1573264401 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f5ba006b380 0x7f5ba006d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:32.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.271+0000 7f5bc7e00640 1 --2- 172.21.15.138:0/1573264401 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f5ba006b380 0x7f5ba006d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:32.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.272+0000 7f5bc7e00640 1 --2- 172.21.15.138:0/1573264401 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f5ba006b380 0x7f5ba006d840 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5bc81599f0 tx=0x7f5bb0009240 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:32.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.273+0000 7f5bc4c00640 1 -- 172.21.15.138:0/1573264401 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bb8057c50 con 0x7f5bc80c6620 2024-01-21T22:25:32.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.514+0000 7f5bc4c00640 1 -- 172.21.15.138:0/1573264401 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f5bb8053f90 con 0x7f5bc80c6620 2024-01-21T22:25:32.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.741+0000 7f5bce600640 1 -- 172.21.15.138:0/1573264401 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5bc815aad0 con 0x7f5bc80c6620 2024-01-21T22:25:32.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.782+0000 7f5bc4c00640 1 -- 172.21.15.138:0/1573264401 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5bc815aad0 con 0x7f5bc80c6620 2024-01-21T22:25:32.783 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-01-21T22:25:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.785+0000 7f5bce600640 1 -- 172.21.15.138:0/1573264401 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f5ba006b380 msgr2=0x7f5ba006d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.785+0000 7f5bce600640 1 --2- 172.21.15.138:0/1573264401 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f5ba006b380 0x7f5ba006d840 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5bc81599f0 tx=0x7f5bb0009240 comp rx=0 tx=0).stop 2024-01-21T22:25:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.785+0000 7f5bce600640 1 -- 172.21.15.138:0/1573264401 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bc80c6620 msgr2=0x7f5bc8154480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.785+0000 7f5bce600640 1 --2- 172.21.15.138:0/1573264401 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bc80c6620 0x7f5bc8154480 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800c3c0 tx=0x7f5bb800c880 comp rx=0 tx=0).stop 2024-01-21T22:25:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.785+0000 7f5bce600640 1 -- 172.21.15.138:0/1573264401 shutdown_connections 2024-01-21T22:25:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.785+0000 7f5bce600640 1 --2- 172.21.15.138:0/1573264401 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f5ba006b380 0x7f5ba006d840 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.785+0000 7f5bce600640 1 --2- 172.21.15.138:0/1573264401 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5bc8153dd0 0x7f5bc80c47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.785+0000 7f5bce600640 1 --2- 172.21.15.138:0/1573264401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5bc80c6c90 0x7f5bc814b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.785+0000 7f5bce600640 1 --2- 172.21.15.138:0/1573264401 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5bc80c6620 0x7f5bc8154480 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.785+0000 7f5bce600640 1 -- 172.21.15.138:0/1573264401 >> 172.21.15.138:0/1573264401 conn(0x7f5bc80cf100 msgr2=0x7f5bc814d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.786+0000 7f5bce600640 1 -- 172.21.15.138:0/1573264401 shutdown_connections 2024-01-21T22:25:32.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:32.786+0000 7f5bce600640 1 -- 172.21.15.138:0/1573264401 wait complete. 2024-01-21T22:25:32.808 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-21T22:25:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.014+0000 7fcc56a00640 1 Processor -- start 2024-01-21T22:25:33.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.014+0000 7fcc56a00640 1 -- start start 2024-01-21T22:25:33.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.016+0000 7fcc56a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcc50151a30 0x7fcc50157730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:33.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.016+0000 7fcc56a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcc50071bb0 0x7fcc501620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:33.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.016+0000 7fcc56a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc50070590 con 0x7fcc5006b4f0 2024-01-21T22:25:33.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.016+0000 7fcc56a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc5006ac50 con 0x7fcc50151a30 2024-01-21T22:25:33.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.016+0000 7fcc56a00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5006a6f0 con 0x7fcc50071bb0 2024-01-21T22:25:33.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.017+0000 7fcc4fe00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcc50151a30 0x7fcc50157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:33.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.017+0000 7fcc54a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcc50071bb0 0x7fcc501620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:33.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.017+0000 7fcc47200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fcc5006b4f0 0x7fcc500c4750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:48584/0 (socket says 172.21.15.138:48584) 2024-01-21T22:25:33.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.017+0000 7fcc47200640 1 -- 172.21.15.138:0/2539926072 learned_addr learned my addr 172.21.15.138:0/2539926072 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:33.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.020+0000 7fcc4fe00640 1 -- 172.21.15.138:0/2539926072 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcc50071bb0 msgr2=0x7fcc501620b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:33.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.020+0000 7fcc4fe00640 1 --2- 172.21.15.138:0/2539926072 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcc50071bb0 0x7fcc501620b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:33.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.020+0000 7fcc4fe00640 1 -- 172.21.15.138:0/2539926072 >> v1:172.21.15.152:6789/0 conn(0x7fcc5006b4f0 legacy=0x7fcc500c4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:33.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.020+0000 7fcc4fe00640 1 -- 172.21.15.138:0/2539926072 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc50151e10 con 0x7fcc50151a30 2024-01-21T22:25:33.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.022+0000 7fcc4fe00640 1 --2- 172.21.15.138:0/2539926072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc50151a30 0x7fcc50157730 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcc400097a0 tx=0x7fcc4002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e86881c6a8ab12c server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:33.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.027+0000 7fcc4f400640 1 -- 172.21.15.138:0/2539926072 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc40030610 con 0x7fcc50151a30 2024-01-21T22:25:33.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.027+0000 7fcc4f400640 1 -- 172.21.15.138:0/2539926072 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc40030770 con 0x7fcc50151a30 2024-01-21T22:25:33.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.027+0000 7fcc4f400640 1 -- 172.21.15.138:0/2539926072 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc40030b00 con 0x7fcc50151a30 2024-01-21T22:25:33.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.028+0000 7fcc56a00640 1 -- 172.21.15.138:0/2539926072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc50151a30 msgr2=0x7fcc50157730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:33.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.028+0000 7fcc56a00640 1 --2- 172.21.15.138:0/2539926072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc50151a30 0x7fcc50157730 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcc400097a0 tx=0x7fcc4002b960 comp rx=0 tx=0).stop 2024-01-21T22:25:33.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.028+0000 7fcc56a00640 1 -- 172.21.15.138:0/2539926072 shutdown_connections 2024-01-21T22:25:33.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.028+0000 7fcc56a00640 1 --2- 172.21.15.138:0/2539926072 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcc50071bb0 0x7fcc501620b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:33.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.028+0000 7fcc56a00640 1 --2- 172.21.15.138:0/2539926072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc50151a30 0x7fcc50157730 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:33.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.028+0000 7fcc56a00640 1 -- 172.21.15.138:0/2539926072 >> 172.21.15.138:0/2539926072 conn(0x7fcc50072a20 msgr2=0x7fcc50066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:33.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.028+0000 7fcc56a00640 1 -- 172.21.15.138:0/2539926072 shutdown_connections 2024-01-21T22:25:33.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.028+0000 7fcc56a00640 1 -- 172.21.15.138:0/2539926072 wait complete. 2024-01-21T22:25:33.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.030+0000 7fcc56a00640 1 Processor -- start 2024-01-21T22:25:33.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.030+0000 7fcc56a00640 1 -- start start 2024-01-21T22:25:33.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.031+0000 7fcc56a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc5006b4f0 0x7fcc500cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:33.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.031+0000 7fcc56a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcc50071bb0 0x7fcc5013f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:33.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.031+0000 7fcc56a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc50151a30 0x7fcc50141440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:33.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.031+0000 7fcc56a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc501f0870 con 0x7fcc50151a30 2024-01-21T22:25:33.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.031+0000 7fcc56a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc501f0310 con 0x7fcc5006b4f0 2024-01-21T22:25:33.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.031+0000 7fcc56a00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc501f0590 con 0x7fcc50071bb0 2024-01-21T22:25:33.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.032+0000 7fcc54a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc50151a30 0x7fcc50141440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:33.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.032+0000 7fcc47200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcc50071bb0 0x7fcc5013f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:33.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.032+0000 7fcc4fe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc5006b4f0 0x7fcc500cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:33.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.032+0000 7fcc54a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc50151a30 0x7fcc50141440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40190/0 (socket says 172.21.15.138:40190) 2024-01-21T22:25:33.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.032+0000 7fcc54a00640 1 -- 172.21.15.138:0/3593742204 learned_addr learned my addr 172.21.15.138:0/3593742204 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:33.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.036+0000 7fcc47200640 1 -- 172.21.15.138:0/3593742204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc5006b4f0 msgr2=0x7fcc500cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:33.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.036+0000 7fcc47200640 1 --2- 172.21.15.138:0/3593742204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc5006b4f0 0x7fcc500cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:33.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.036+0000 7fcc47200640 1 -- 172.21.15.138:0/3593742204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc50151a30 msgr2=0x7fcc50141440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:33.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.036+0000 7fcc47200640 1 --2- 172.21.15.138:0/3593742204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc50151a30 0x7fcc50141440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:33.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.036+0000 7fcc47200640 1 -- 172.21.15.138:0/3593742204 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc40009400 con 0x7fcc50071bb0 2024-01-21T22:25:33.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.038+0000 7fcc47200640 1 --2- 172.21.15.138:0/3593742204 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcc50071bb0 0x7fcc5013f8a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcc500c23d0 tx=0x7fcc3800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:33.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.043+0000 7fcc4cc00640 1 -- 172.21.15.138:0/3593742204 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc38005ce0 con 0x7fcc50071bb0 2024-01-21T22:25:33.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.044+0000 7fcc4cc00640 1 -- 172.21.15.138:0/3593742204 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc38005e60 con 0x7fcc50071bb0 2024-01-21T22:25:33.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.044+0000 7fcc56a00640 1 -- 172.21.15.138:0/3593742204 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc501401f0 con 0x7fcc50071bb0 2024-01-21T22:25:33.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.044+0000 7fcc4cc00640 1 -- 172.21.15.138:0/3593742204 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc38007310 con 0x7fcc50071bb0 2024-01-21T22:25:33.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.044+0000 7fcc56a00640 1 -- 172.21.15.138:0/3593742204 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc50155c40 con 0x7fcc50071bb0 2024-01-21T22:25:33.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.045+0000 7fcc56a00640 1 -- 172.21.15.138:0/3593742204 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc5004eb10 con 0x7fcc50071bb0 2024-01-21T22:25:33.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.064+0000 7fcc4cc00640 1 -- 172.21.15.138:0/3593742204 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fcc38007470 con 0x7fcc50071bb0 2024-01-21T22:25:33.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.065+0000 7fcc4cc00640 1 --2- 172.21.15.138:0/3593742204 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcc2806b3b0 0x7fcc2806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:33.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.065+0000 7fcc4fe00640 1 --2- 172.21.15.138:0/3593742204 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcc2806b3b0 0x7fcc2806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:33.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.066+0000 7fcc4cc00640 1 -- 172.21.15.138:0/3593742204 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc38007720 con 0x7fcc50071bb0 2024-01-21T22:25:33.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.066+0000 7fcc4fe00640 1 --2- 172.21.15.138:0/3593742204 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcc2806b3b0 0x7fcc2806d870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcc40036000 tx=0x7fcc4003b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:33.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.313+0000 7fcc4cc00640 1 -- 172.21.15.138:0/3593742204 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fcc380541f0 con 0x7fcc50071bb0 2024-01-21T22:25:33.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.525+0000 7fcc56a00640 1 -- 172.21.15.138:0/3593742204 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcc500c8760 con 0x7fcc50071bb0 2024-01-21T22:25:33.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.531+0000 7fcc4cc00640 1 -- 172.21.15.138:0/3593742204 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcc38017020 con 0x7fcc50071bb0 2024-01-21T22:25:33.533 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 32 2024-01-21T22:25:33.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.534+0000 7fcc56a00640 1 -- 172.21.15.138:0/3593742204 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcc2806b3b0 msgr2=0x7fcc2806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:33.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.534+0000 7fcc56a00640 1 --2- 172.21.15.138:0/3593742204 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcc2806b3b0 0x7fcc2806d870 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcc40036000 tx=0x7fcc4003b040 comp rx=0 tx=0).stop 2024-01-21T22:25:33.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.534+0000 7fcc56a00640 1 -- 172.21.15.138:0/3593742204 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcc50071bb0 msgr2=0x7fcc5013f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:33.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.534+0000 7fcc56a00640 1 --2- 172.21.15.138:0/3593742204 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcc50071bb0 0x7fcc5013f8a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcc500c23d0 tx=0x7fcc3800c800 comp rx=0 tx=0).stop 2024-01-21T22:25:33.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.534+0000 7fcc56a00640 1 -- 172.21.15.138:0/3593742204 shutdown_connections 2024-01-21T22:25:33.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.534+0000 7fcc56a00640 1 --2- 172.21.15.138:0/3593742204 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcc2806b3b0 0x7fcc2806d870 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:33.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.534+0000 7fcc56a00640 1 --2- 172.21.15.138:0/3593742204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc50151a30 0x7fcc50141440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:33.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.535+0000 7fcc56a00640 1 --2- 172.21.15.138:0/3593742204 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcc50071bb0 0x7fcc5013f8a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:33.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.535+0000 7fcc56a00640 1 --2- 172.21.15.138:0/3593742204 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcc5006b4f0 0x7fcc500cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:33.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.535+0000 7fcc56a00640 1 -- 172.21.15.138:0/3593742204 >> 172.21.15.138:0/3593742204 conn(0x7fcc50072a20 msgr2=0x7fcc50070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:33.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.535+0000 7fcc56a00640 1 -- 172.21.15.138:0/3593742204 shutdown_connections 2024-01-21T22:25:33.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.535+0000 7fcc56a00640 1 -- 172.21.15.138:0/3593742204 wait complete. 2024-01-21T22:25:33.554 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-21T22:25:33.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.759+0000 7f155b400640 1 Processor -- start 2024-01-21T22:25:33.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.759+0000 7f155b400640 1 -- start start 2024-01-21T22:25:33.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.760+0000 7f155b400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f15540c9bd0 0x7f15540ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:33.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.760+0000 7f155b400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f15540c7e30 0x7f15540cae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:33.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.760+0000 7f155b400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15540c7290 con 0x7f1554153c60 2024-01-21T22:25:33.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.760+0000 7f155b400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f155415a570 con 0x7f15540c9bd0 2024-01-21T22:25:33.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.760+0000 7f155b400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15540cb640 con 0x7f15540c7e30 2024-01-21T22:25:33.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.765+0000 7f1559400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f15540c7e30 0x7f15540cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:33.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.765+0000 7f1559400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f15540c7e30 0x7f15540cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:40826/0 (socket says 172.21.15.138:40826) 2024-01-21T22:25:33.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.765+0000 7f1559400640 1 -- 172.21.15.138:0/3599462860 learned_addr learned my addr 172.21.15.138:0/3599462860 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:33.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.766+0000 7f154bc00640 1 --2- 172.21.15.138:0/3599462860 >> v2:172.21.15.152:3300/0 conn(0x7f15540c9bd0 0x7f15540ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:33.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.768+0000 7f1559400640 1 -- 172.21.15.138:0/3599462860 >> v2:172.21.15.152:3300/0 conn(0x7f15540c9bd0 msgr2=0x7f15540ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:33.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.768+0000 7f1559400640 1 --2- 172.21.15.138:0/3599462860 >> v2:172.21.15.152:3300/0 conn(0x7f15540c9bd0 0x7f15540ca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:33.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.768+0000 7f1559400640 1 -- 172.21.15.138:0/3599462860 >> v1:172.21.15.138:6789/0 conn(0x7f1554153c60 legacy=0x7f15540c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:33.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.768+0000 7f1559400640 1 -- 172.21.15.138:0/3599462860 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15540ca110 con 0x7f15540c7e30 2024-01-21T22:25:33.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.772+0000 7f1559400640 1 --2- 172.21.15.138:0/3599462860 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f15540c7e30 0x7f15540cae00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f154c009e50 tx=0x7f154c02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb19dbfdf10ff23e server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:33.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.776+0000 7f1553e00640 1 -- 172.21.15.138:0/3599462860 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c02ca70 con 0x7f15540c7e30 2024-01-21T22:25:33.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.776+0000 7f1553e00640 1 -- 172.21.15.138:0/3599462860 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154c02cbd0 con 0x7f15540c7e30 2024-01-21T22:25:33.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.776+0000 7f1553e00640 1 -- 172.21.15.138:0/3599462860 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154c0343f0 con 0x7f15540c7e30 2024-01-21T22:25:33.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.777+0000 7f155b400640 1 -- 172.21.15.138:0/3599462860 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f15540c7e30 msgr2=0x7f15540cae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:33.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.777+0000 7f155b400640 1 --2- 172.21.15.138:0/3599462860 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f15540c7e30 0x7f15540cae00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f154c009e50 tx=0x7f154c02bed0 comp rx=0 tx=0).stop 2024-01-21T22:25:33.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.777+0000 7f155b400640 1 -- 172.21.15.138:0/3599462860 shutdown_connections 2024-01-21T22:25:33.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.777+0000 7f155b400640 1 --2- 172.21.15.138:0/3599462860 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f15540c7e30 0x7f15540cae00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:33.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.777+0000 7f155b400640 1 --2- 172.21.15.138:0/3599462860 >> v2:172.21.15.152:3300/0 conn(0x7f15540c9bd0 0x7f15540ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:33.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.777+0000 7f155b400640 1 -- 172.21.15.138:0/3599462860 >> 172.21.15.138:0/3599462860 conn(0x7f15540cf430 msgr2=0x7f15540cf840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:33.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.777+0000 7f155b400640 1 -- 172.21.15.138:0/3599462860 shutdown_connections 2024-01-21T22:25:33.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.777+0000 7f155b400640 1 -- 172.21.15.138:0/3599462860 wait complete. 2024-01-21T22:25:33.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.780+0000 7f155b400640 1 Processor -- start 2024-01-21T22:25:33.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.780+0000 7f155b400640 1 -- start start 2024-01-21T22:25:33.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.781+0000 7f155b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15540c7e30 0x7f1554156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:33.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.781+0000 7f155b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15540c9bd0 0x7f15541efe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:33.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.781+0000 7f155b400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1554153c60 0x7f15541f0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:33.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.781+0000 7f155b400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1554156290 con 0x7f15540c7e30 2024-01-21T22:25:33.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.781+0000 7f155b400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15541553a0 con 0x7f15540c9bd0 2024-01-21T22:25:33.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.781+0000 7f155b400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1554155680 con 0x7f1554153c60 2024-01-21T22:25:33.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.782+0000 7f1559400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1554153c60 0x7f15541f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:33.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.782+0000 7f154bc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15540c9bd0 0x7f15541efe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:33.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.782+0000 7f154bc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15540c9bd0 0x7f15541efe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:51440/0 (socket says 172.21.15.138:51440) 2024-01-21T22:25:33.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.782+0000 7f1559400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1554153c60 0x7f15541f0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:40828/0 (socket says 172.21.15.138:40828) 2024-01-21T22:25:33.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.782+0000 7f154bc00640 1 -- 172.21.15.138:0/3753151240 learned_addr learned my addr 172.21.15.138:0/3753151240 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:33.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.785+0000 7f154bc00640 1 -- 172.21.15.138:0/3753151240 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1554153c60 msgr2=0x7f15541f0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:33.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.785+0000 7f154bc00640 1 --2- 172.21.15.138:0/3753151240 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1554153c60 0x7f15541f0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:33.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.785+0000 7f154bc00640 1 -- 172.21.15.138:0/3753151240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15540c7e30 msgr2=0x7f1554156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:33.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.785+0000 7f154bc00640 1 --2- 172.21.15.138:0/3753151240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15540c7e30 0x7f1554156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:33.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.785+0000 7f154bc00640 1 -- 172.21.15.138:0/3753151240 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c009b30 con 0x7f15540c9bd0 2024-01-21T22:25:33.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.788+0000 7f154bc00640 1 --2- 172.21.15.138:0/3753151240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15540c9bd0 0x7f15541efe20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f153c00e9b0 tx=0x7f153c00ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:33.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.791+0000 7f1551600640 1 -- 172.21.15.138:0/3753151240 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c009670 con 0x7f15540c9bd0 2024-01-21T22:25:33.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.791+0000 7f1551600640 1 -- 172.21.15.138:0/3753151240 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153c00c9a0 con 0x7f15540c9bd0 2024-01-21T22:25:33.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.791+0000 7f1551600640 1 -- 172.21.15.138:0/3753151240 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153c00cd50 con 0x7f15540c9bd0 2024-01-21T22:25:33.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.791+0000 7f155b400640 1 -- 172.21.15.138:0/3753151240 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1554155960 con 0x7f15540c9bd0 2024-01-21T22:25:33.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.792+0000 7f155b400640 1 -- 172.21.15.138:0/3753151240 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1554155f20 con 0x7f15540c9bd0 2024-01-21T22:25:33.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.793+0000 7f155b400640 1 -- 172.21.15.138:0/3753151240 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15541560e0 con 0x7f15540c9bd0 2024-01-21T22:25:33.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.815+0000 7f1551600640 1 -- 172.21.15.138:0/3753151240 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f153c013470 con 0x7f15540c9bd0 2024-01-21T22:25:33.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.816+0000 7f1551600640 1 --2- 172.21.15.138:0/3753151240 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f152c06b380 0x7f152c06d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:33.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.816+0000 7f1551600640 1 -- 172.21.15.138:0/3753151240 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f153c08a5c0 con 0x7f15540c9bd0 2024-01-21T22:25:33.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.817+0000 7f1558a00640 1 --2- 172.21.15.138:0/3753151240 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f152c06b380 0x7f152c06d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:33.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:33.818+0000 7f1558a00640 1 --2- 172.21.15.138:0/3753151240 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f152c06b380 0x7f152c06d840 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1544009650 tx=0x7f1544006ba0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:34.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.052+0000 7f1551600640 1 -- 172.21.15.138:0/3753151240 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f153c055250 con 0x7f15540c9bd0 2024-01-21T22:25:34.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.262+0000 7f155b400640 1 -- 172.21.15.138:0/3753151240 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f155404eb10 con 0x7f15540c9bd0 2024-01-21T22:25:34.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.269+0000 7f1551600640 1 -- 172.21.15.138:0/3753151240 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f153c0170a0 con 0x7f15540c9bd0 2024-01-21T22:25:34.270 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 64 2024-01-21T22:25:34.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.272+0000 7f155b400640 1 -- 172.21.15.138:0/3753151240 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f152c06b380 msgr2=0x7f152c06d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:34.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.272+0000 7f155b400640 1 --2- 172.21.15.138:0/3753151240 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f152c06b380 0x7f152c06d840 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1544009650 tx=0x7f1544006ba0 comp rx=0 tx=0).stop 2024-01-21T22:25:34.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.272+0000 7f155b400640 1 -- 172.21.15.138:0/3753151240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15540c9bd0 msgr2=0x7f15541efe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:34.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.272+0000 7f155b400640 1 --2- 172.21.15.138:0/3753151240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15540c9bd0 0x7f15541efe20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f153c00e9b0 tx=0x7f153c00ee70 comp rx=0 tx=0).stop 2024-01-21T22:25:34.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.272+0000 7f155b400640 1 -- 172.21.15.138:0/3753151240 shutdown_connections 2024-01-21T22:25:34.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.272+0000 7f155b400640 1 --2- 172.21.15.138:0/3753151240 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f152c06b380 0x7f152c06d840 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:34.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.272+0000 7f155b400640 1 --2- 172.21.15.138:0/3753151240 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1554153c60 0x7f15541f0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:34.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.272+0000 7f155b400640 1 --2- 172.21.15.138:0/3753151240 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f15540c9bd0 0x7f15541efe20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:34.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.272+0000 7f155b400640 1 --2- 172.21.15.138:0/3753151240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f15540c7e30 0x7f1554156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:34.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.272+0000 7f155b400640 1 -- 172.21.15.138:0/3753151240 >> 172.21.15.138:0/3753151240 conn(0x7f15540cf430 msgr2=0x7f15540cf810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:34.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.272+0000 7f155b400640 1 -- 172.21.15.138:0/3753151240 shutdown_connections 2024-01-21T22:25:34.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.272+0000 7f155b400640 1 -- 172.21.15.138:0/3753151240 wait complete. 2024-01-21T22:25:34.293 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-21T22:25:34.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.499+0000 7fee62a00640 1 Processor -- start 2024-01-21T22:25:34.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.499+0000 7fee62a00640 1 -- start start 2024-01-21T22:25:34.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.500+0000 7fee62a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fee5c151e10 0x7fee5c15ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:34.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.500+0000 7fee62a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee5c15b3a0 0x7fee5c0c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:34.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.500+0000 7fee62a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee5c1583d0 con 0x7fee5c151a30 2024-01-21T22:25:34.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.500+0000 7fee62a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee5c06c7b0 con 0x7fee5c151e10 2024-01-21T22:25:34.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.500+0000 7fee62a00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c157890 con 0x7fee5c15b3a0 2024-01-21T22:25:34.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.501+0000 7fee60a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee5c15b3a0 0x7fee5c0c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:34.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.501+0000 7fee53e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fee5c151e10 0x7fee5c15ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:34.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.501+0000 7fee5be00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fee5c151a30 0x7fee5c0c5ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:48596/0 (socket says 172.21.15.138:48596) 2024-01-21T22:25:34.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.501+0000 7fee5be00640 1 -- 172.21.15.138:0/1684065031 learned_addr learned my addr 172.21.15.138:0/1684065031 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:34.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.504+0000 7fee5b400640 1 -- 172.21.15.138:0/1684065031 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792456773 0 0) 0x7fee5c1583d0 con 0x7fee5c151a30 2024-01-21T22:25:34.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.504+0000 7fee60a00640 1 -- 172.21.15.138:0/1684065031 >> v2:172.21.15.138:3300/0 conn(0x7fee5c151e10 msgr2=0x7fee5c15ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:34.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.504+0000 7fee60a00640 1 --2- 172.21.15.138:0/1684065031 >> v2:172.21.15.138:3300/0 conn(0x7fee5c151e10 0x7fee5c15ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:34.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.504+0000 7fee60a00640 1 -- 172.21.15.138:0/1684065031 >> v1:172.21.15.152:6789/0 conn(0x7fee5c151a30 legacy=0x7fee5c0c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:34.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.504+0000 7fee60a00640 1 -- 172.21.15.138:0/1684065031 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c06c4d0 con 0x7fee5c15b3a0 2024-01-21T22:25:34.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.506+0000 7fee60a00640 1 --2- 172.21.15.138:0/1684065031 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee5c15b3a0 0x7fee5c0c47c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fee54009d90 tx=0x7fee5402bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f2caf66e0aed797 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:34.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.510+0000 7fee5b400640 1 -- 172.21.15.138:0/1684065031 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5402cce0 con 0x7fee5c15b3a0 2024-01-21T22:25:34.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.510+0000 7fee5b400640 1 -- 172.21.15.138:0/1684065031 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5402ce40 con 0x7fee5c15b3a0 2024-01-21T22:25:34.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.510+0000 7fee5b400640 1 -- 172.21.15.138:0/1684065031 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee54034660 con 0x7fee5c15b3a0 2024-01-21T22:25:34.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.511+0000 7fee62a00640 1 -- 172.21.15.138:0/1684065031 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee5c15b3a0 msgr2=0x7fee5c0c47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:34.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.511+0000 7fee62a00640 1 --2- 172.21.15.138:0/1684065031 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee5c15b3a0 0x7fee5c0c47c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fee54009d90 tx=0x7fee5402bf30 comp rx=0 tx=0).stop 2024-01-21T22:25:34.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.511+0000 7fee62a00640 1 -- 172.21.15.138:0/1684065031 shutdown_connections 2024-01-21T22:25:34.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.511+0000 7fee62a00640 1 --2- 172.21.15.138:0/1684065031 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee5c15b3a0 0x7fee5c0c47c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:34.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.511+0000 7fee62a00640 1 --2- 172.21.15.138:0/1684065031 >> v2:172.21.15.138:3300/0 conn(0x7fee5c151e10 0x7fee5c15ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:34.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.511+0000 7fee62a00640 1 -- 172.21.15.138:0/1684065031 >> 172.21.15.138:0/1684065031 conn(0x7fee5c0cb0b0 msgr2=0x7fee5c0cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:34.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.511+0000 7fee62a00640 1 -- 172.21.15.138:0/1684065031 shutdown_connections 2024-01-21T22:25:34.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.511+0000 7fee62a00640 1 -- 172.21.15.138:0/1684065031 wait complete. 2024-01-21T22:25:34.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.513+0000 7fee62a00640 1 Processor -- start 2024-01-21T22:25:34.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.513+0000 7fee62a00640 1 -- start start 2024-01-21T22:25:34.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.514+0000 7fee62a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee5c151a30 0x7fee5c1e2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:34.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.514+0000 7fee62a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee5c15b3a0 0x7fee5c0b7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:34.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.514+0000 7fee62a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee5c154ea0 0x7fee5c1ede30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:34.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.514+0000 7fee62a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c0c1170 con 0x7fee5c151a30 2024-01-21T22:25:34.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.514+0000 7fee62a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c1e29f0 con 0x7fee5c15b3a0 2024-01-21T22:25:34.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.514+0000 7fee62a00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5c0c0eb0 con 0x7fee5c154ea0 2024-01-21T22:25:34.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.514+0000 7fee60a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee5c154ea0 0x7fee5c1ede30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:34.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.514+0000 7fee5be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee5c151a30 0x7fee5c1e2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:34.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.514+0000 7fee53e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee5c15b3a0 0x7fee5c0b7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:34.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.515+0000 7fee5be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee5c151a30 0x7fee5c1e2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40220/0 (socket says 172.21.15.138:40220) 2024-01-21T22:25:34.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.515+0000 7fee5be00640 1 -- 172.21.15.138:0/2221539785 learned_addr learned my addr 172.21.15.138:0/2221539785 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:34.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.518+0000 7fee60a00640 1 -- 172.21.15.138:0/2221539785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee5c15b3a0 msgr2=0x7fee5c0b7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:34.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.518+0000 7fee60a00640 1 --2- 172.21.15.138:0/2221539785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee5c15b3a0 0x7fee5c0b7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:34.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.518+0000 7fee60a00640 1 -- 172.21.15.138:0/2221539785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee5c151a30 msgr2=0x7fee5c1e2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:34.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.518+0000 7fee60a00640 1 --2- 172.21.15.138:0/2221539785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee5c151a30 0x7fee5c1e2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:34.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.518+0000 7fee60a00640 1 -- 172.21.15.138:0/2221539785 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee54009a70 con 0x7fee5c154ea0 2024-01-21T22:25:34.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.520+0000 7fee60a00640 1 --2- 172.21.15.138:0/2221539785 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee5c154ea0 0x7fee5c1ede30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fee5c1579d0 tx=0x7fee54004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:34.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.524+0000 7fee58c00640 1 -- 172.21.15.138:0/2221539785 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5403d600 con 0x7fee5c154ea0 2024-01-21T22:25:34.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.524+0000 7fee58c00640 1 -- 172.21.15.138:0/2221539785 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5403d760 con 0x7fee5c154ea0 2024-01-21T22:25:34.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.524+0000 7fee58c00640 1 -- 172.21.15.138:0/2221539785 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5403daf0 con 0x7fee5c154ea0 2024-01-21T22:25:34.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.524+0000 7fee62a00640 1 -- 172.21.15.138:0/2221539785 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5c1e20c0 con 0x7fee5c154ea0 2024-01-21T22:25:34.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.524+0000 7fee62a00640 1 -- 172.21.15.138:0/2221539785 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee5c1e2680 con 0x7fee5c154ea0 2024-01-21T22:25:34.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.525+0000 7fee62a00640 1 -- 172.21.15.138:0/2221539785 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee5c158840 con 0x7fee5c154ea0 2024-01-21T22:25:34.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.546+0000 7fee58c00640 1 -- 172.21.15.138:0/2221539785 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fee5403b040 con 0x7fee5c154ea0 2024-01-21T22:25:34.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.547+0000 7fee58c00640 1 --2- 172.21.15.138:0/2221539785 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fee3406b380 0x7fee3406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:34.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.548+0000 7fee5be00640 1 --2- 172.21.15.138:0/2221539785 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fee3406b380 0x7fee3406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:34.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.548+0000 7fee58c00640 1 -- 172.21.15.138:0/2221539785 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee5407a970 con 0x7fee5c154ea0 2024-01-21T22:25:34.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.548+0000 7fee5be00640 1 --2- 172.21.15.138:0/2221539785 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fee3406b380 0x7fee3406d840 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fee4c000f60 tx=0x7fee4c007e10 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:34.670 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:25:34.678 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:25:34.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:34.790+0000 7fee58c00640 1 -- 172.21.15.138:0/2221539785 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fee54076cb0 con 0x7fee5c154ea0 2024-01-21T22:25:35.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.026+0000 7fee62a00640 1 -- 172.21.15.138:0/2221539785 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee5c158840 con 0x7fee5c154ea0 2024-01-21T22:25:35.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.038+0000 7fee58c00640 1 -- 172.21.15.138:0/2221539785 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+6530 (secure 0 0 0) 0x7fee5c158840 con 0x7fee5c154ea0 2024-01-21T22:25:35.040 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:25:35.040 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/3112363661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":3112363661},{"type":"v1","addr":"172.21.15.152:6837","nonce":3112363661}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4223,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/3222259940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":3222259940},{"type":"v1","addr":"172.21.15.152:6839","nonce":3222259940}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/909784319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":909784319},{"type":"v1","addr":"172.21.15.138:6833","nonce":909784319}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4229,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3318550662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3318550662},{"type":"v1","addr":"172.21.15.138:6835","nonce":3318550662}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2986531362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2986531362},{"type":"v1","addr":"172.21.15.138:6837","nonce":2986531362}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-21T22:25:04.967250+0000","modified":"2024-01-21T22:25:12.598970+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":5,"state":"up:active","state_seq":5,"addr":"172.21.15.152:6835/2663179600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2663179600},{"type":"v1","addr":"172.21.15.152:6835","nonce":2663179600}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-21T22:25:35.040 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 10 2024-01-21T22:25:35.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.041+0000 7fee62a00640 1 -- 172.21.15.138:0/2221539785 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fee3406b380 msgr2=0x7fee3406d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:35.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.041+0000 7fee62a00640 1 --2- 172.21.15.138:0/2221539785 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fee3406b380 0x7fee3406d840 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fee4c000f60 tx=0x7fee4c007e10 comp rx=0 tx=0).stop 2024-01-21T22:25:35.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.041+0000 7fee62a00640 1 -- 172.21.15.138:0/2221539785 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee5c154ea0 msgr2=0x7fee5c1ede30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:35.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.041+0000 7fee62a00640 1 --2- 172.21.15.138:0/2221539785 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee5c154ea0 0x7fee5c1ede30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fee5c1579d0 tx=0x7fee54004a40 comp rx=0 tx=0).stop 2024-01-21T22:25:35.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.041+0000 7fee62a00640 1 -- 172.21.15.138:0/2221539785 shutdown_connections 2024-01-21T22:25:35.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.041+0000 7fee62a00640 1 --2- 172.21.15.138:0/2221539785 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fee3406b380 0x7fee3406d840 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:35.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.041+0000 7fee62a00640 1 --2- 172.21.15.138:0/2221539785 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee5c154ea0 0x7fee5c1ede30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:35.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.041+0000 7fee62a00640 1 --2- 172.21.15.138:0/2221539785 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee5c15b3a0 0x7fee5c0b7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:35.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.041+0000 7fee62a00640 1 --2- 172.21.15.138:0/2221539785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee5c151a30 0x7fee5c1e2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:35.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.042+0000 7fee62a00640 1 -- 172.21.15.138:0/2221539785 >> 172.21.15.138:0/2221539785 conn(0x7fee5c0cb0b0 msgr2=0x7fee5c0cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:35.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.042+0000 7fee62a00640 1 -- 172.21.15.138:0/2221539785 shutdown_connections 2024-01-21T22:25:35.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.042+0000 7fee62a00640 1 -- 172.21.15.138:0/2221539785 wait complete. 2024-01-21T22:25:35.064 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-21T22:25:35.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.264+0000 7fe8ea000640 1 Processor -- start 2024-01-21T22:25:35.265 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.264+0000 7fe8ea000640 1 -- start start 2024-01-21T22:25:35.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.266+0000 7fe8ea000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe8e40cdce0 0x7fe8e40c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:35.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.266+0000 7fe8ea000640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e40c8020 con 0x7fe8e40cd900 2024-01-21T22:25:35.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.266+0000 7fe8ea000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e415d090 con 0x7fe8e41536c0 2024-01-21T22:25:35.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.266+0000 7fe8ea000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8e415cea0 con 0x7fe8e40cdce0 2024-01-21T22:25:35.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.266+0000 7fe8e2a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe8e40cdce0 0x7fe8e40c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:35.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.267+0000 7fe8e2a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe8e40cdce0 0x7fe8e40c8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:51468/0 (socket says 172.21.15.138:51468) 2024-01-21T22:25:35.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.267+0000 7fe8e2a00640 1 -- 172.21.15.138:0/1830895776 learned_addr learned my addr 172.21.15.138:0/1830895776 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:35.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.271+0000 7fe8e2a00640 1 -- 172.21.15.138:0/1830895776 >> v1:172.21.15.152:6789/0 conn(0x7fe8e41536c0 legacy=0x7fe8e4159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:35.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.271+0000 7fe8e2a00640 1 -- 172.21.15.138:0/1830895776 >> v1:172.21.15.138:6789/0 conn(0x7fe8e40cd900 legacy=0x7fe8e406f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:35.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.271+0000 7fe8e2a00640 1 -- 172.21.15.138:0/1830895776 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e415d370 con 0x7fe8e40cdce0 2024-01-21T22:25:35.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.274+0000 7fe8e2a00640 1 --2- 172.21.15.138:0/1830895776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8e40cdce0 0x7fe8e40c8c20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc009870 tx=0x7fe8cc02b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84f76bc28bac01a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:35.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.277+0000 7fe8e2000640 1 -- 172.21.15.138:0/1830895776 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc030860 con 0x7fe8e40cdce0 2024-01-21T22:25:35.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.278+0000 7fe8e2000640 1 -- 172.21.15.138:0/1830895776 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8cc0309c0 con 0x7fe8e40cdce0 2024-01-21T22:25:35.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.278+0000 7fe8e2000640 1 -- 172.21.15.138:0/1830895776 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc030d50 con 0x7fe8e40cdce0 2024-01-21T22:25:35.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.279+0000 7fe8ea000640 1 -- 172.21.15.138:0/1830895776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8e40cdce0 msgr2=0x7fe8e40c8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:35.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.279+0000 7fe8ea000640 1 --2- 172.21.15.138:0/1830895776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8e40cdce0 0x7fe8e40c8c20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc009870 tx=0x7fe8cc02b9a0 comp rx=0 tx=0).stop 2024-01-21T22:25:35.280 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.279+0000 7fe8ea000640 1 -- 172.21.15.138:0/1830895776 shutdown_connections 2024-01-21T22:25:35.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.279+0000 7fe8ea000640 1 --2- 172.21.15.138:0/1830895776 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8e40cdce0 0x7fe8e40c8c20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:35.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.279+0000 7fe8ea000640 1 -- 172.21.15.138:0/1830895776 >> 172.21.15.138:0/1830895776 conn(0x7fe8e4066820 msgr2=0x7fe8e4066c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:35.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.279+0000 7fe8ea000640 1 -- 172.21.15.138:0/1830895776 shutdown_connections 2024-01-21T22:25:35.281 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.279+0000 7fe8ea000640 1 -- 172.21.15.138:0/1830895776 wait complete. 2024-01-21T22:25:35.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.281+0000 7fe8ea000640 1 Processor -- start 2024-01-21T22:25:35.282 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.281+0000 7fe8ea000640 1 -- start start 2024-01-21T22:25:35.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.282+0000 7fe8ea000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe8e40cd900 0x7fe8e41e30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:35.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.282+0000 7fe8ea000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8e40cdce0 0x7fe8e4152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:35.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.282+0000 7fe8ea000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8e41536c0 0x7fe8e41f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:35.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.282+0000 7fe8ea000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e41e3600 con 0x7fe8e41536c0 2024-01-21T22:25:35.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.282+0000 7fe8ea000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e41f34c0 con 0x7fe8e40cdce0 2024-01-21T22:25:35.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.282+0000 7fe8ea000640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e41f31e0 con 0x7fe8e40cd900 2024-01-21T22:25:35.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.283+0000 7fe8e3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8e41536c0 0x7fe8e41f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:35.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.283+0000 7fe8e3400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe8e40cd900 0x7fe8e41e30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:35.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.283+0000 7fe8e2a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8e40cdce0 0x7fe8e4152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:35.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.283+0000 7fe8e3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8e41536c0 0x7fe8e41f25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40236/0 (socket says 172.21.15.138:40236) 2024-01-21T22:25:35.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.283+0000 7fe8e3e00640 1 -- 172.21.15.138:0/2521043073 learned_addr learned my addr 172.21.15.138:0/2521043073 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:35.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.287+0000 7fe8e2a00640 1 -- 172.21.15.138:0/2521043073 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe8e40cd900 msgr2=0x7fe8e41e30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:35.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.287+0000 7fe8e2a00640 1 --2- 172.21.15.138:0/2521043073 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe8e40cd900 0x7fe8e41e30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:35.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.287+0000 7fe8e2a00640 1 -- 172.21.15.138:0/2521043073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8e41536c0 msgr2=0x7fe8e41f25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:35.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.287+0000 7fe8e2a00640 1 --2- 172.21.15.138:0/2521043073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8e41536c0 0x7fe8e41f25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:35.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.287+0000 7fe8e2a00640 1 -- 172.21.15.138:0/2521043073 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8cc0094d0 con 0x7fe8e40cdce0 2024-01-21T22:25:35.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.289+0000 7fe8e2a00640 1 --2- 172.21.15.138:0/2521043073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8e40cdce0 0x7fe8e4152120 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc033000 tx=0x7fe8cc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:35.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.293+0000 7fe8db400640 1 -- 172.21.15.138:0/2521043073 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc043400 con 0x7fe8e40cdce0 2024-01-21T22:25:35.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.294+0000 7fe8db400640 1 -- 172.21.15.138:0/2521043073 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8cc043560 con 0x7fe8e40cdce0 2024-01-21T22:25:35.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.294+0000 7fe8db400640 1 -- 172.21.15.138:0/2521043073 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc043910 con 0x7fe8e40cdce0 2024-01-21T22:25:35.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.294+0000 7fe8ea000640 1 -- 172.21.15.138:0/2521043073 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e4153ca0 con 0x7fe8e40cdce0 2024-01-21T22:25:35.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.294+0000 7fe8ea000640 1 -- 172.21.15.138:0/2521043073 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e40c7de0 con 0x7fe8e40cdce0 2024-01-21T22:25:35.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.295+0000 7fe8ea000640 1 -- 172.21.15.138:0/2521043073 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e404eaa0 con 0x7fe8e40cdce0 2024-01-21T22:25:35.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.316+0000 7fe8db400640 1 -- 172.21.15.138:0/2521043073 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fe8cc043a70 con 0x7fe8e40cdce0 2024-01-21T22:25:35.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.317+0000 7fe8db400640 1 --2- 172.21.15.138:0/2521043073 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe8bc06b380 0x7fe8bc06d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:35.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.318+0000 7fe8e3400640 1 --2- 172.21.15.138:0/2521043073 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe8bc06b380 0x7fe8bc06d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:35.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.318+0000 7fe8db400640 1 -- 172.21.15.138:0/2521043073 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8cc03e070 con 0x7fe8e40cdce0 2024-01-21T22:25:35.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.319+0000 7fe8e3400640 1 --2- 172.21.15.138:0/2521043073 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe8bc06b380 0x7fe8bc06d840 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40017f0 tx=0x7fe8d4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:35.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.573+0000 7fe8db400640 1 -- 172.21.15.138:0/2521043073 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe8cc0785f0 con 0x7fe8e40cdce0 2024-01-21T22:25:35.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.776+0000 7fe8ea000640 1 -- 172.21.15.138:0/2521043073 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8e406c180 con 0x7fe8e40cdce0 2024-01-21T22:25:35.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.810+0000 7fe8db400640 1 -- 172.21.15.138:0/2521043073 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16522 (secure 0 0 0) 0x7fe8cc034070 con 0x7fe8e40cdce0 2024-01-21T22:25:35.812 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:25:35.812 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":23,"fsid":"d488d446-149f-48d0-89ca-b2830ae72846","created":"2024-01-21T22:24:28.581932+0000","modified":"2024-01-21T22:25:06.475790+0000","last_up_change":"2024-01-21T22:24:47.145799+0000","last_in_change":"2024-01-21T22:24:37.672807+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-21T22:24:49.586178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-21T22:25:02.044454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-21T22:25:03.361765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d812b2d6-c8a3-4015-bfdb-5766b25eae5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6825","nonce":2849203277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6827","nonce":2849203277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6831","nonce":2849203277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6829","nonce":2849203277}]},"public_addr":"172.21.15.138:6825/2849203277","cluster_addr":"172.21.15.138:6827/2849203277","heartbeat_back_addr":"172.21.15.138:6831/2849203277","heartbeat_front_addr":"172.21.15.138:6829/2849203277","state":["exists","up"]},{"osd":1,"uuid":"d8d875df-c840-46cc-b6c0-a847f9c93999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6817","nonce":174521731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6819","nonce":174521731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6823","nonce":174521731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6821","nonce":174521731}]},"public_addr":"172.21.15.138:6817/174521731","cluster_addr":"172.21.15.138:6819/174521731","heartbeat_back_addr":"172.21.15.138:6823/174521731","heartbeat_front_addr":"172.21.15.138:6821/174521731","state":["exists","up"]},{"osd":2,"uuid":"4bd94373-76ca-47a6-870b-16d96d7c9e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6801","nonce":1365838748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6803","nonce":1365838748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6809","nonce":1365838748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6805","nonce":1365838748}]},"public_addr":"172.21.15.138:6801/1365838748","cluster_addr":"172.21.15.138:6803/1365838748","heartbeat_back_addr":"172.21.15.138:6809/1365838748","heartbeat_front_addr":"172.21.15.138:6805/1365838748","state":["exists","up"]},{"osd":3,"uuid":"07267087-d4d2-4e88-ab9b-bb87f6dd061a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6808","nonce":1841873885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6811","nonce":1841873885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6815","nonce":1841873885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6813","nonce":1841873885}]},"public_addr":"172.21.15.138:6808/1841873885","cluster_addr":"172.21.15.138:6811/1841873885","heartbeat_back_addr":"172.21.15.138:6815/1841873885","heartbeat_front_addr":"172.21.15.138:6813/1841873885","state":["exists","up"]},{"osd":4,"uuid":"b47ef758-d783-4168-a9a5-11db3de4ca12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6827","nonce":3960812298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6829","nonce":3960812298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6833","nonce":3960812298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6831","nonce":3960812298}]},"public_addr":"172.21.15.152:6827/3960812298","cluster_addr":"172.21.15.152:6829/3960812298","heartbeat_back_addr":"172.21.15.152:6833/3960812298","heartbeat_front_addr":"172.21.15.152:6831/3960812298","state":["exists","up"]},{"osd":5,"uuid":"0ea5d43e-61d8-4823-815f-9e42c62fae97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6819","nonce":4095234314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6821","nonce":4095234314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6825","nonce":4095234314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6823","nonce":4095234314}]},"public_addr":"172.21.15.152:6819/4095234314","cluster_addr":"172.21.15.152:6821/4095234314","heartbeat_back_addr":"172.21.15.152:6825/4095234314","heartbeat_front_addr":"172.21.15.152:6823/4095234314","state":["exists","up"]},{"osd":6,"uuid":"59ffac36-e2e0-4693-ac82-ad5eccf6224d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6803","nonce":223848127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6805","nonce":223848127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6809","nonce":223848127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6807","nonce":223848127}]},"public_addr":"172.21.15.152:6803/223848127","cluster_addr":"172.21.15.152:6805/223848127","heartbeat_back_addr":"172.21.15.152:6809/223848127","heartbeat_front_addr":"172.21.15.152:6807/223848127","state":["exists","up"]},{"osd":7,"uuid":"4f892c7d-32a5-4960-bf88-4ed7c863cd0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6811","nonce":52509095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6813","nonce":52509095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6817","nonce":52509095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6815","nonce":52509095}]},"public_addr":"172.21.15.152:6811/52509095","cluster_addr":"172.21.15.152:6813/52509095","heartbeat_back_addr":"172.21.15.152:6817/52509095","heartbeat_front_addr":"172.21.15.152:6815/52509095","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.434460+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.374709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.613134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.298148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.242144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.198102+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.093385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.197180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-21T22:25:35.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.813+0000 7fe8ea000640 1 -- 172.21.15.138:0/2521043073 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe8bc06b380 msgr2=0x7fe8bc06d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:35.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.813+0000 7fe8ea000640 1 --2- 172.21.15.138:0/2521043073 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe8bc06b380 0x7fe8bc06d840 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe8d40017f0 tx=0x7fe8d4009040 comp rx=0 tx=0).stop 2024-01-21T22:25:35.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.813+0000 7fe8ea000640 1 -- 172.21.15.138:0/2521043073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8e40cdce0 msgr2=0x7fe8e4152120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:35.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.813+0000 7fe8ea000640 1 --2- 172.21.15.138:0/2521043073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8e40cdce0 0x7fe8e4152120 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc033000 tx=0x7fe8cc004730 comp rx=0 tx=0).stop 2024-01-21T22:25:35.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.813+0000 7fe8ea000640 1 -- 172.21.15.138:0/2521043073 shutdown_connections 2024-01-21T22:25:35.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.813+0000 7fe8ea000640 1 --2- 172.21.15.138:0/2521043073 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe8bc06b380 0x7fe8bc06d840 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:35.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.813+0000 7fe8ea000640 1 --2- 172.21.15.138:0/2521043073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8e41536c0 0x7fe8e41f25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:35.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.813+0000 7fe8ea000640 1 --2- 172.21.15.138:0/2521043073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe8e40cdce0 0x7fe8e4152120 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:35.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.813+0000 7fe8ea000640 1 --2- 172.21.15.138:0/2521043073 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe8e40cd900 0x7fe8e41e30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:35.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.813+0000 7fe8ea000640 1 -- 172.21.15.138:0/2521043073 >> 172.21.15.138:0/2521043073 conn(0x7fe8e4066820 msgr2=0x7fe8e406c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:35.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.814+0000 7fe8ea000640 1 -- 172.21.15.138:0/2521043073 shutdown_connections 2024-01-21T22:25:35.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:35.814+0000 7fe8ea000640 1 -- 172.21.15.138:0/2521043073 wait complete. 2024-01-21T22:25:35.833 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-21T22:25:36.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.032+0000 7f5fa4400640 1 Processor -- start 2024-01-21T22:25:36.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.032+0000 7f5fa4400640 1 -- start start 2024-01-21T22:25:36.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.035+0000 7f5fa4400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5f9c150a00 0x7f5f9c0c25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:36.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.035+0000 7f5fa4400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f5f9c06ad40 0x7f5f9c06b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:36.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.035+0000 7f5fa4400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f9c157740 con 0x7f5f9c0716e0 2024-01-21T22:25:36.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.035+0000 7f5fa4400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f9c0c4cc0 con 0x7f5f9c06ad40 2024-01-21T22:25:36.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.035+0000 7f5fa4400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f9c06a920 con 0x7f5f9c150a00 2024-01-21T22:25:36.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.036+0000 7f5fa1a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5f9c150a00 0x7f5f9c0c25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:36.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.036+0000 7f5fa1a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f5f9c150a00 0x7f5f9c0c25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:51492/0 (socket says 172.21.15.138:51492) 2024-01-21T22:25:36.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.036+0000 7f5fa1a00640 1 -- 172.21.15.138:0/3262452124 learned_addr learned my addr 172.21.15.138:0/3262452124 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:36.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.038+0000 7f5fa1a00640 1 -- 172.21.15.138:0/3262452124 >> v2:172.21.15.138:3300/0 conn(0x7f5f9c06ad40 msgr2=0x7f5f9c06b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:36.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.038+0000 7f5fa1a00640 1 --2- 172.21.15.138:0/3262452124 >> v2:172.21.15.138:3300/0 conn(0x7f5f9c06ad40 0x7f5f9c06b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:36.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.038+0000 7f5fa1a00640 1 -- 172.21.15.138:0/3262452124 >> v1:172.21.15.138:6789/0 conn(0x7f5f9c0716e0 legacy=0x7f5f9c0c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:36.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.038+0000 7f5fa1a00640 1 -- 172.21.15.138:0/3262452124 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c06b6b0 con 0x7f5f9c150a00 2024-01-21T22:25:36.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.040+0000 7f5fa1a00640 1 --2- 172.21.15.138:0/3262452124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f9c150a00 0x7f5f9c0c25b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c06b840 tx=0x7f5f8c02da90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e03ea436e040b84 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:36.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.044+0000 7f5f9be00640 1 -- 172.21.15.138:0/3262452124 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c02f850 con 0x7f5f9c150a00 2024-01-21T22:25:36.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.044+0000 7f5f9be00640 1 -- 172.21.15.138:0/3262452124 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8c02f9b0 con 0x7f5f9c150a00 2024-01-21T22:25:36.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.044+0000 7f5f9be00640 1 -- 172.21.15.138:0/3262452124 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c02fd40 con 0x7f5f9c150a00 2024-01-21T22:25:36.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.045+0000 7f5fa4400640 1 -- 172.21.15.138:0/3262452124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f9c150a00 msgr2=0x7f5f9c0c25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:36.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.045+0000 7f5fa4400640 1 --2- 172.21.15.138:0/3262452124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f9c150a00 0x7f5f9c0c25b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c06b840 tx=0x7f5f8c02da90 comp rx=0 tx=0).stop 2024-01-21T22:25:36.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.045+0000 7f5fa4400640 1 -- 172.21.15.138:0/3262452124 shutdown_connections 2024-01-21T22:25:36.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.045+0000 7f5fa4400640 1 --2- 172.21.15.138:0/3262452124 >> v2:172.21.15.138:3300/0 conn(0x7f5f9c06ad40 0x7f5f9c06b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:36.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.045+0000 7f5fa4400640 1 --2- 172.21.15.138:0/3262452124 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f9c150a00 0x7f5f9c0c25b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:36.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.045+0000 7f5fa4400640 1 -- 172.21.15.138:0/3262452124 >> 172.21.15.138:0/3262452124 conn(0x7f5f9c0cb0b0 msgr2=0x7f5f9c0cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:36.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.046+0000 7f5fa4400640 1 -- 172.21.15.138:0/3262452124 shutdown_connections 2024-01-21T22:25:36.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.046+0000 7f5fa4400640 1 -- 172.21.15.138:0/3262452124 wait complete. 2024-01-21T22:25:36.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.047+0000 7f5fa4400640 1 Processor -- start 2024-01-21T22:25:36.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.047+0000 7f5fa4400640 1 -- start start 2024-01-21T22:25:36.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.049+0000 7f5fa4400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f9c06ad40 0x7f5f9c1e5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:36.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.049+0000 7f5fa4400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f9c0716e0 0x7f5f9c14d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:36.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.049+0000 7f5fa4400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f9c150a00 0x7f5f9c146a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:36.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.049+0000 7f5fa4400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c1ef950 con 0x7f5f9c06ad40 2024-01-21T22:25:36.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.049+0000 7f5fa4400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c1efb60 con 0x7f5f9c150a00 2024-01-21T22:25:36.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.049+0000 7f5fa4400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c1efe40 con 0x7f5f9c0716e0 2024-01-21T22:25:36.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.050+0000 7f5fa1000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f9c0716e0 0x7f5f9c14d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:36.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.050+0000 7f5fa1000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f9c0716e0 0x7f5f9c14d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:40858/0 (socket says 172.21.15.138:40858) 2024-01-21T22:25:36.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.050+0000 7f5fa1000640 1 -- 172.21.15.138:0/25025142 learned_addr learned my addr 172.21.15.138:0/25025142 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:36.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.052+0000 7f5fa1a00640 1 --2- 172.21.15.138:0/25025142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f9c06ad40 0x7f5f9c1e5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:36.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.052+0000 7f5fa2400640 1 --2- 172.21.15.138:0/25025142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f9c150a00 0x7f5f9c146a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:36.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.054+0000 7f5fa1000640 1 -- 172.21.15.138:0/25025142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f9c150a00 msgr2=0x7f5f9c146a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:36.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.054+0000 7f5fa1000640 1 --2- 172.21.15.138:0/25025142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f9c150a00 0x7f5f9c146a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:36.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.054+0000 7f5fa1000640 1 -- 172.21.15.138:0/25025142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f9c06ad40 msgr2=0x7f5f9c1e5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:36.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.054+0000 7f5fa1000640 1 --2- 172.21.15.138:0/25025142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f9c06ad40 0x7f5f9c1e5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:36.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.054+0000 7f5fa1000640 1 -- 172.21.15.138:0/25025142 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8c0094d0 con 0x7f5f9c0716e0 2024-01-21T22:25:36.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.060+0000 7f5fa1000640 1 --2- 172.21.15.138:0/25025142 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f9c0716e0 0x7f5f9c14d6a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400c410 tx=0x7f5f8400c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:36.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.064+0000 7f5f99600640 1 -- 172.21.15.138:0/25025142 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f840035f0 con 0x7f5f9c0716e0 2024-01-21T22:25:36.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.064+0000 7f5f99600640 1 -- 172.21.15.138:0/25025142 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f84003750 con 0x7f5f9c0716e0 2024-01-21T22:25:36.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.064+0000 7f5fa4400640 1 -- 172.21.15.138:0/25025142 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f9c1f0120 con 0x7f5f9c0716e0 2024-01-21T22:25:36.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.064+0000 7f5f99600640 1 -- 172.21.15.138:0/25025142 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f84003b40 con 0x7f5f9c0716e0 2024-01-21T22:25:36.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.064+0000 7f5fa4400640 1 -- 172.21.15.138:0/25025142 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9c1f06e0 con 0x7f5f9c0716e0 2024-01-21T22:25:36.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.065+0000 7f5fa4400640 1 -- 172.21.15.138:0/25025142 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9c06b170 con 0x7f5f9c0716e0 2024-01-21T22:25:36.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.087+0000 7f5f99600640 1 -- 172.21.15.138:0/25025142 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f5f84003ca0 con 0x7f5f9c0716e0 2024-01-21T22:25:36.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.089+0000 7f5f99600640 1 --2- 172.21.15.138:0/25025142 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f5f7406b380 0x7f5f7406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:36.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.089+0000 7f5f99600640 1 -- 172.21.15.138:0/25025142 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f84089580 con 0x7f5f9c0716e0 2024-01-21T22:25:36.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.090+0000 7f5fa1a00640 1 --2- 172.21.15.138:0/25025142 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f5f7406b380 0x7f5f7406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:36.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.091+0000 7f5fa1a00640 1 --2- 172.21.15.138:0/25025142 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f5f7406b380 0x7f5f7406d840 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c038000 tx=0x7f5f8c008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:36.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.332+0000 7f5f99600640 1 -- 172.21.15.138:0/25025142 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f5f840543a0 con 0x7f5f9c0716e0 2024-01-21T22:25:36.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.584+0000 7f5fa4400640 1 -- 172.21.15.138:0/25025142 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f9c14e8a0 con 0x7f5f9c0716e0 2024-01-21T22:25:36.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.702+0000 7f5f99600640 1 -- 172.21.15.138:0/25025142 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+6530 (secure 0 0 0) 0x7f5f840579c0 con 0x7f5f9c0716e0 2024-01-21T22:25:36.704 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:25:36.704 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/3112363661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":3112363661},{"type":"v1","addr":"172.21.15.152:6837","nonce":3112363661}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4223,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/3222259940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":3222259940},{"type":"v1","addr":"172.21.15.152:6839","nonce":3222259940}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/909784319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":909784319},{"type":"v1","addr":"172.21.15.138:6833","nonce":909784319}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4229,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3318550662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3318550662},{"type":"v1","addr":"172.21.15.138:6835","nonce":3318550662}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2986531362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2986531362},{"type":"v1","addr":"172.21.15.138:6837","nonce":2986531362}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-21T22:25:04.967250+0000","modified":"2024-01-21T22:25:12.598970+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":5,"state":"up:active","state_seq":5,"addr":"172.21.15.152:6835/2663179600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2663179600},{"type":"v1","addr":"172.21.15.152:6835","nonce":2663179600}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-21T22:25:36.704 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 10 2024-01-21T22:25:36.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.706+0000 7f5fa4400640 1 -- 172.21.15.138:0/25025142 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f5f7406b380 msgr2=0x7f5f7406d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:36.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.706+0000 7f5fa4400640 1 --2- 172.21.15.138:0/25025142 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f5f7406b380 0x7f5f7406d840 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c038000 tx=0x7f5f8c008030 comp rx=0 tx=0).stop 2024-01-21T22:25:36.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.706+0000 7f5fa4400640 1 -- 172.21.15.138:0/25025142 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f9c0716e0 msgr2=0x7f5f9c14d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:36.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.706+0000 7f5fa4400640 1 --2- 172.21.15.138:0/25025142 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f9c0716e0 0x7f5f9c14d6a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400c410 tx=0x7f5f8400c8d0 comp rx=0 tx=0).stop 2024-01-21T22:25:36.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.706+0000 7f5fa4400640 1 -- 172.21.15.138:0/25025142 shutdown_connections 2024-01-21T22:25:36.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.706+0000 7f5fa4400640 1 --2- 172.21.15.138:0/25025142 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f5f7406b380 0x7f5f7406d840 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:36.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.706+0000 7f5fa4400640 1 --2- 172.21.15.138:0/25025142 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f5f9c150a00 0x7f5f9c146a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:36.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.706+0000 7f5fa4400640 1 --2- 172.21.15.138:0/25025142 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f5f9c0716e0 0x7f5f9c14d6a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:36.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.706+0000 7f5fa4400640 1 --2- 172.21.15.138:0/25025142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f9c06ad40 0x7f5f9c1e5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:36.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.706+0000 7f5fa4400640 1 -- 172.21.15.138:0/25025142 >> 172.21.15.138:0/25025142 conn(0x7f5f9c0cb0b0 msgr2=0x7f5f9c14f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:36.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.706+0000 7f5fa4400640 1 -- 172.21.15.138:0/25025142 shutdown_connections 2024-01-21T22:25:36.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:36.706+0000 7f5fa4400640 1 -- 172.21.15.138:0/25025142 wait complete. 2024-01-21T22:25:37.334 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 10, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-01-21T22:25:04.967250+0000', 'modified': '2024-01-21T22:25:12.598970+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4276}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4276': {'gid': 4276, 'name': 'd', 'rank': 0, 'incarnation': 5, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.152:6835/2663179600', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.152:6834', 'nonce': 2663179600}, {'type': 'v1', 'addr': '172.21.15.152:6835', 'nonce': 2663179600}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701272860131327, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-01-21T22:25:37.334 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-01-21T22:25:37.334 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-01-21T22:25:37.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.549+0000 7fdaa7a00640 1 Processor -- start 2024-01-21T22:25:37.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.549+0000 7fdaa7a00640 1 -- start start 2024-01-21T22:25:37.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.550+0000 7fdaa7a00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdaa006c5d0 0x7fdaa00c6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:37.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.550+0000 7fdaa7a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdaa006c9b0 0x7fdaa00c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:37.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.550+0000 7fdaa7a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa00c55a0 con 0x7fdaa00c5970 2024-01-21T22:25:37.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.550+0000 7fdaa7a00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaa00c3d30 con 0x7fdaa006c5d0 2024-01-21T22:25:37.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.550+0000 7fdaa7a00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa00c4310 con 0x7fdaa006c9b0 2024-01-21T22:25:37.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.551+0000 7fda97c00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdaa006c9b0 0x7fdaa00c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:37.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.551+0000 7fdaa5000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdaa006c5d0 0x7fdaa00c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:37.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.552+0000 7fdaa5000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fdaa006c5d0 0x7fdaa00c6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:51514/0 (socket says 172.21.15.138:51514) 2024-01-21T22:25:37.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.552+0000 7fdaa5000640 1 -- 172.21.15.138:0/3427003904 learned_addr learned my addr 172.21.15.138:0/3427003904 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:37.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.555+0000 7fda97c00640 1 -- 172.21.15.138:0/3427003904 >> v2:172.21.15.152:3300/0 conn(0x7fdaa006c5d0 msgr2=0x7fdaa00c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:37.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.555+0000 7fda97c00640 1 --2- 172.21.15.138:0/3427003904 >> v2:172.21.15.152:3300/0 conn(0x7fdaa006c5d0 0x7fdaa00c6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:37.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.555+0000 7fda97c00640 1 -- 172.21.15.138:0/3427003904 >> v1:172.21.15.152:6789/0 conn(0x7fdaa00c5970 legacy=0x7fdaa006d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:37.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.555+0000 7fda97c00640 1 -- 172.21.15.138:0/3427003904 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa00c28d0 con 0x7fdaa006c9b0 2024-01-21T22:25:37.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.557+0000 7fda97c00640 1 --2- 172.21.15.138:0/3427003904 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdaa006c9b0 0x7fdaa00c5060 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fda880097a0 tx=0x7fda8802b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=520e890a285515a8 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:37.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.562+0000 7fda9fe00640 1 -- 172.21.15.138:0/3427003904 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda88030800 con 0x7fdaa006c9b0 2024-01-21T22:25:37.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.562+0000 7fda9fe00640 1 -- 172.21.15.138:0/3427003904 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda88030960 con 0x7fdaa006c9b0 2024-01-21T22:25:37.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.562+0000 7fda9fe00640 1 -- 172.21.15.138:0/3427003904 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda88030cf0 con 0x7fdaa006c9b0 2024-01-21T22:25:37.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.562+0000 7fdaa7a00640 1 -- 172.21.15.138:0/3427003904 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdaa006c9b0 msgr2=0x7fdaa00c5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:37.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.562+0000 7fdaa7a00640 1 --2- 172.21.15.138:0/3427003904 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdaa006c9b0 0x7fdaa00c5060 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fda880097a0 tx=0x7fda8802b940 comp rx=0 tx=0).stop 2024-01-21T22:25:37.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.562+0000 7fdaa7a00640 1 -- 172.21.15.138:0/3427003904 shutdown_connections 2024-01-21T22:25:37.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.562+0000 7fdaa7a00640 1 --2- 172.21.15.138:0/3427003904 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdaa006c9b0 0x7fdaa00c5060 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:37.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.562+0000 7fdaa7a00640 1 --2- 172.21.15.138:0/3427003904 >> v2:172.21.15.152:3300/0 conn(0x7fdaa006c5d0 0x7fdaa00c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:37.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.562+0000 7fdaa7a00640 1 -- 172.21.15.138:0/3427003904 >> 172.21.15.138:0/3427003904 conn(0x7fdaa00cb070 msgr2=0x7fdaa00cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:37.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.563+0000 7fdaa7a00640 1 -- 172.21.15.138:0/3427003904 shutdown_connections 2024-01-21T22:25:37.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.563+0000 7fdaa7a00640 1 -- 172.21.15.138:0/3427003904 wait complete. 2024-01-21T22:25:37.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.564+0000 7fdaa7a00640 1 Processor -- start 2024-01-21T22:25:37.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.565+0000 7fdaa7a00640 1 -- start start 2024-01-21T22:25:37.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.566+0000 7fdaa7a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdaa006c5d0 0x7fdaa00ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:37.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.566+0000 7fdaa7a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdaa006c9b0 0x7fdaa0156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:37.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.566+0000 7fdaa7a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdaa00c5970 0x7fdaa01ef6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:37.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.566+0000 7fdaa7a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa00ade30 con 0x7fdaa00c5970 2024-01-21T22:25:37.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.566+0000 7fdaa7a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa00ae080 con 0x7fdaa006c5d0 2024-01-21T22:25:37.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.566+0000 7fdaa7a00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa00af770 con 0x7fdaa006c9b0 2024-01-21T22:25:37.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.567+0000 7fdaa5000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdaa006c5d0 0x7fdaa00ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:37.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.567+0000 7fda97c00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdaa006c9b0 0x7fdaa0156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:37.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.568+0000 7fda97c00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdaa006c9b0 0x7fdaa0156ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:40890/0 (socket says 172.21.15.138:40890) 2024-01-21T22:25:37.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.568+0000 7fda97c00640 1 -- 172.21.15.138:0/22597936 learned_addr learned my addr 172.21.15.138:0/22597936 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:37.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.571+0000 7fdaa5a00640 1 --2- 172.21.15.138:0/22597936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdaa00c5970 0x7fdaa01ef6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:37.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.572+0000 7fda97c00640 1 -- 172.21.15.138:0/22597936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdaa006c5d0 msgr2=0x7fdaa00ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:37.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.572+0000 7fda97c00640 1 --2- 172.21.15.138:0/22597936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdaa006c5d0 0x7fdaa00ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:37.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.572+0000 7fda97c00640 1 -- 172.21.15.138:0/22597936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdaa00c5970 msgr2=0x7fdaa01ef6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:37.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.572+0000 7fda97c00640 1 --2- 172.21.15.138:0/22597936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdaa00c5970 0x7fdaa01ef6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:37.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.573+0000 7fda97c00640 1 -- 172.21.15.138:0/22597936 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda88009400 con 0x7fdaa006c9b0 2024-01-21T22:25:37.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.573+0000 7fdaa5a00640 1 --2- 172.21.15.138:0/22597936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdaa00c5970 0x7fdaa01ef6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-01-21T22:25:37.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.578+0000 7fda97c00640 1 --2- 172.21.15.138:0/22597936 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdaa006c9b0 0x7fdaa0156ce0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fda8802bf50 tx=0x7fda88004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:37.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.582+0000 7fda9d600640 1 -- 172.21.15.138:0/22597936 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8803e040 con 0x7fdaa006c9b0 2024-01-21T22:25:37.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.582+0000 7fda9d600640 1 -- 172.21.15.138:0/22597936 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda88039cb0 con 0x7fdaa006c9b0 2024-01-21T22:25:37.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.582+0000 7fda9d600640 1 -- 172.21.15.138:0/22597936 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda88044630 con 0x7fdaa006c9b0 2024-01-21T22:25:37.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.582+0000 7fdaa7a00640 1 -- 172.21.15.138:0/22597936 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa00afa50 con 0x7fdaa006c9b0 2024-01-21T22:25:37.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.583+0000 7fdaa7a00640 1 -- 172.21.15.138:0/22597936 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa00b0010 con 0x7fdaa006c9b0 2024-01-21T22:25:37.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.587+0000 7fdaaa824740 1 Processor -- start 2024-01-21T22:25:37.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.587+0000 7fdaaa824740 1 -- start start 2024-01-21T22:25:37.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.588+0000 7fdaaa824740 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x5569fc64b190 0x5569fc64b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:37.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.588+0000 7fdaaa824740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc695010 0x5569fc64bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:37.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.588+0000 7fdaaa824740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fc649bf0 0x5569fc68ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:37.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.588+0000 7fdaaa824740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569fc5b7d80 con 0x5569fc649bf0 2024-01-21T22:25:37.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.588+0000 7fdaaa824740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569fc5b5470 con 0x5569fc695010 2024-01-21T22:25:37.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.588+0000 7fdaaa824740 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569fc5b0d60 con 0x5569fc64b190 2024-01-21T22:25:37.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.589+0000 7fdaa5000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc695010 0x5569fc64bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:37.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.589+0000 7fdaa5000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc695010 0x5569fc64bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:51536/0 (socket says 172.21.15.138:51536) 2024-01-21T22:25:37.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.589+0000 7fdaa5000640 1 -- 172.21.15.138:0/2046850904 learned_addr learned my addr 172.21.15.138:0/2046850904 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:37.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.595+0000 7fdaa5000640 1 -- 172.21.15.138:0/2046850904 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x5569fc64b190 msgr2=0x5569fc64b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:37.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.595+0000 7fdaa5000640 1 --2- 172.21.15.138:0/2046850904 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x5569fc64b190 0x5569fc64b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:37.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.595+0000 7fdaa5000640 1 -- 172.21.15.138:0/2046850904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fc649bf0 msgr2=0x5569fc68ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:37.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.595+0000 7fdaa5000640 1 --2- 172.21.15.138:0/2046850904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fc649bf0 0x5569fc68ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:37.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.595+0000 7fdaa5000640 1 -- 172.21.15.138:0/2046850904 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569fc566280 con 0x5569fc695010 2024-01-21T22:25:37.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.601+0000 7fdaa5000640 1 --2- 172.21.15.138:0/2046850904 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc695010 0x5569fc64bab0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fda90009ca0 tx=0x7fda90011690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84cff9dcd0fc631 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:37.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.611+0000 7fda9d600640 1 -- 172.21.15.138:0/22597936 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fda8804d3b0 con 0x7fdaa006c9b0 2024-01-21T22:25:37.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.612+0000 7fda9d600640 1 --2- 172.21.15.138:0/22597936 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fda7806b380 0x7fda7806d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:37.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.612+0000 7fdaa5a00640 1 --2- 172.21.15.138:0/22597936 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fda7806b380 0x7fda7806d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:37.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.613+0000 7fdaa5a00640 1 --2- 172.21.15.138:0/22597936 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fda7806b380 0x7fda7806d840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fda98007fd0 tx=0x7fda98009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:37.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.614+0000 7fda9d600640 1 -- 172.21.15.138:0/22597936 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fda88039e20 con 0x7fdaa006c9b0 2024-01-21T22:25:37.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.660+0000 7fda95400640 1 -- 172.21.15.138:0/2046850904 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90011ec0 con 0x5569fc695010 2024-01-21T22:25:37.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.661+0000 7fda95400640 1 -- 172.21.15.138:0/2046850904 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda90014440 con 0x5569fc695010 2024-01-21T22:25:37.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.661+0000 7fda95400640 1 -- 172.21.15.138:0/2046850904 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda900147b0 con 0x5569fc695010 2024-01-21T22:25:37.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.662+0000 7fdaaa824740 1 -- 172.21.15.138:0/2046850904 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc695010 msgr2=0x5569fc64bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:37.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.662+0000 7fdaaa824740 1 --2- 172.21.15.138:0/2046850904 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc695010 0x5569fc64bab0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fda90009ca0 tx=0x7fda90011690 comp rx=0 tx=0).stop 2024-01-21T22:25:37.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.662+0000 7fdaaa824740 1 -- 172.21.15.138:0/2046850904 shutdown_connections 2024-01-21T22:25:37.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.662+0000 7fdaaa824740 1 --2- 172.21.15.138:0/2046850904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fc649bf0 0x5569fc68ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:37.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.662+0000 7fdaaa824740 1 --2- 172.21.15.138:0/2046850904 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc695010 0x5569fc64bab0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:37.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.662+0000 7fdaaa824740 1 --2- 172.21.15.138:0/2046850904 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x5569fc64b190 0x5569fc64b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:37.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.662+0000 7fdaaa824740 1 -- 172.21.15.138:0/2046850904 >> 172.21.15.138:0/2046850904 conn(0x5569fc647b90 msgr2=0x5569fc64a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:37.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.662+0000 7fdaaa824740 1 -- 172.21.15.138:0/2046850904 shutdown_connections 2024-01-21T22:25:37.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.662+0000 7fdaaa824740 1 -- 172.21.15.138:0/2046850904 wait complete. 2024-01-21T22:25:37.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.663+0000 7fdaaa824740 1 -- start start 2024-01-21T22:25:37.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.664+0000 7fdaaa824740 1 Processor -- start 2024-01-21T22:25:37.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.665+0000 7fdaaa824740 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x5569fc64b190 0x5569fc6710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:37.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.665+0000 7fdaaa824740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc695010 0x5569fc673810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:37.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.665+0000 7fdaaa824740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fc673d50 0x5569fc688bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:37.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.665+0000 7fdaaa824740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569fc54c740 con 0x5569fc673d50 2024-01-21T22:25:37.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.665+0000 7fdaaa824740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569fc56a0c0 con 0x5569fc695010 2024-01-21T22:25:37.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.665+0000 7fdaaa824740 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569fc312f20 con 0x5569fc64b190 2024-01-21T22:25:37.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.665+0000 7fdaaa824740 10 client.0 _ll_register_callbacks cb 0x5569fc56e800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-21T22:25:37.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.667+0000 7fda97c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc695010 0x5569fc673810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:37.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.667+0000 7fda97c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc695010 0x5569fc673810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:51550/0 (socket says 172.21.15.138:51550) 2024-01-21T22:25:37.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.667+0000 7fda97c00640 1 -- 172.21.15.138:0/2675897349 learned_addr learned my addr 172.21.15.138:0/2675897349 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:37.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.670+0000 7fda97c00640 1 -- 172.21.15.138:0/2675897349 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x5569fc64b190 msgr2=0x5569fc6710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:37.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.670+0000 7fda97c00640 1 --2- 172.21.15.138:0/2675897349 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x5569fc64b190 0x5569fc6710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:37.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.670+0000 7fda97c00640 1 -- 172.21.15.138:0/2675897349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fc673d50 msgr2=0x5569fc688bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:37.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.670+0000 7fda97c00640 1 --2- 172.21.15.138:0/2675897349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fc673d50 0x5569fc688bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:37.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.670+0000 7fda97c00640 1 -- 172.21.15.138:0/2675897349 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda90009930 con 0x5569fc695010 2024-01-21T22:25:37.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.675+0000 7fda97c00640 1 --2- 172.21.15.138:0/2675897349 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc695010 0x5569fc673810 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fda880aa890 tx=0x7fda88009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b18bc1b3d69325e server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:37.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.675+0000 7fda8e000640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-01-21T22:25:37.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.681+0000 7fda8e000640 1 -- 172.21.15.138:0/2675897349 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda88092170 con 0x5569fc695010 2024-01-21T22:25:37.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.681+0000 7fda8e000640 1 -- 172.21.15.138:0/2675897349 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fda880922d0 con 0x5569fc695010 2024-01-21T22:25:37.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.681+0000 7fdaaa824740 1 -- 172.21.15.138:0/2675897349 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5569fc5a9bf0 con 0x5569fc695010 2024-01-21T22:25:37.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.758+0000 7fda8e000640 1 -- 172.21.15.138:0/2675897349 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=10) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda8807faf0 con 0x5569fc695010 2024-01-21T22:25:37.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.758+0000 7fdaaa824740 10 client.4600 fetch_fsmap learned FSMap version 10 2024-01-21T22:25:37.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.758+0000 7fdaaa824740 1 -- 172.21.15.138:0/2675897349 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=10,monmap=2+}) v3 -- 0x5569fc5af3e0 con 0x5569fc695010 2024-01-21T22:25:37.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.771+0000 7fda8e000640 1 -- 172.21.15.138:0/2675897349 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5569fc5a9bf0 con 0x5569fc695010 2024-01-21T22:25:37.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.771+0000 7fda8e000640 1 -- 172.21.15.138:0/2675897349 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 10) v1 ==== 4444+0+0 (secure 0 0 0) 0x7fda8807fc50 con 0x5569fc695010 2024-01-21T22:25:37.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.771+0000 7fdaaa824740 10 client.4600 fetch_fsmap finished waiting for FSMap version 10 2024-01-21T22:25:37.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.771+0000 7fdaaa824740 10 client.4600 resolve_mds: resolved name 'd' to daemon mds.d 2024-01-21T22:25:37.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.771+0000 7fdaaa824740 20 client.4600 populate_metadata read hostname 'smithi138' 2024-01-21T22:25:37.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.771+0000 7fdaaa824740 1 --2- 172.21.15.138:0/2675897349 >> [v2:172.21.15.152:6834/2663179600,v1:172.21.15.152:6835/2663179600] conn(0x5569fc69f6d0 0x5569fc69fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:37.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.771+0000 7fdaaa824740 4 client.4600 mds_command: new command op to 4276 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-01-21T22:25:37.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.771+0000 7fdaaa824740 1 -- 172.21.15.138:0/2675897349 --> [v2:172.21.15.152:6834/2663179600,v1:172.21.15.152:6835/2663179600] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569fc5a9910 con 0x5569fc69f6d0 2024-01-21T22:25:37.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.781+0000 7fdaa5000640 1 --2- 172.21.15.138:0/2675897349 >> [v2:172.21.15.152:6834/2663179600,v1:172.21.15.152:6835/2663179600] conn(0x5569fc69f6d0 0x5569fc69fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:37.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.865+0000 7fdaa5000640 1 --2- 172.21.15.138:0/2675897349 >> [v2:172.21.15.152:6834/2663179600,v1:172.21.15.152:6835/2663179600] conn(0x5569fc69f6d0 0x5569fc69fab0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=453a7decf159fb1e server_cookie=1d615e59516d7229 in_seq=0 out_seq=0 2024-01-21T22:25:37.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.865+0000 7fda8e000640 10 client.4600 ms_handle_connect on v2:172.21.15.152:6834/2663179600 2024-01-21T22:25:37.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.981+0000 7fda8e000640 1 -- 172.21.15.138:0/2675897349 <== mds.0 v2:172.21.15.152:6834/2663179600 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5569fc5a9910 con 0x5569fc69f6d0 2024-01-21T22:25:37.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.981+0000 7fda8e000640 10 client.4600 handle_command_reply: tid=0 2024-01-21T22:25:37.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.981+0000 7fdaaa824740 1 client.4600 shutdown 2024-01-21T22:25:37.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.981+0000 7fdaaa824740 1 -- 172.21.15.138:0/2675897349 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc695010 msgr2=0x5569fc673810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:37.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.981+0000 7fdaaa824740 1 --2- 172.21.15.138:0/2675897349 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc695010 0x5569fc673810 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fda880aa890 tx=0x7fda88009250 comp rx=0 tx=0).stop 2024-01-21T22:25:37.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.982+0000 7fdaaa824740 1 -- 172.21.15.138:0/2675897349 shutdown_connections 2024-01-21T22:25:37.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.982+0000 7fdaaa824740 1 --2- 172.21.15.138:0/2675897349 >> [v2:172.21.15.152:6834/2663179600,v1:172.21.15.152:6835/2663179600] conn(0x5569fc69f6d0 0x5569fc69fab0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:37.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.982+0000 7fdaaa824740 1 --2- 172.21.15.138:0/2675897349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fc673d50 0x5569fc688bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:37.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.982+0000 7fdaaa824740 1 --2- 172.21.15.138:0/2675897349 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc695010 0x5569fc673810 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:37.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.982+0000 7fda8e000640 0 client.4600 ms_handle_reset on v2:172.21.15.152:6834/2663179600 2024-01-21T22:25:37.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.982+0000 7fdaaa824740 1 --2- 172.21.15.138:0/2675897349 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x5569fc64b190 0x5569fc6710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:37.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.982+0000 7fdaaa824740 1 -- 172.21.15.138:0/2675897349 >> 172.21.15.138:0/2675897349 conn(0x5569fc649bf0 msgr2=0x5569fc649270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:37.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.982+0000 7fdaaa824740 1 -- 172.21.15.138:0/2675897349 shutdown_connections 2024-01-21T22:25:37.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.982+0000 7fdaaa824740 1 -- 172.21.15.138:0/2675897349 wait complete. 2024-01-21T22:25:37.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:37.983+0000 7fdaaa824740 20 client.4600 trim_cache size 0 max 16384 2024-01-21T22:25:38.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.031+0000 7fdaaa824740 1 Processor -- start 2024-01-21T22:25:38.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.031+0000 7fdaaa824740 1 -- start start 2024-01-21T22:25:38.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.033+0000 7fdaaa824740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc64b190 0x5569fc644990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:38.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.033+0000 7fdaaa824740 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x5569fc673d50 0x5569fc644ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:38.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.033+0000 7fdaaa824740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fc695010 0x5569fc66dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:38.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.033+0000 7fdaaa824740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569fc648020 con 0x5569fc695010 2024-01-21T22:25:38.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.033+0000 7fdaaa824740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569fc689a10 con 0x5569fc64b190 2024-01-21T22:25:38.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.033+0000 7fdaaa824740 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569fc5e3150 con 0x5569fc673d50 2024-01-21T22:25:38.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.033+0000 7fdaa5000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc64b190 0x5569fc644990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:38.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.034+0000 7fda97c00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x5569fc673d50 0x5569fc644ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:38.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.034+0000 7fdaa5000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc64b190 0x5569fc644990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:58348/0 (socket says 172.21.15.138:58348) 2024-01-21T22:25:38.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.034+0000 7fdaa5000640 1 -- 172.21.15.138:0/612385695 learned_addr learned my addr 172.21.15.138:0/612385695 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:38.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.034+0000 7fdaa5a00640 1 --2- 172.21.15.138:0/612385695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fc695010 0x5569fc66dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:38.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.036+0000 7fdaa5000640 1 -- 172.21.15.138:0/612385695 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x5569fc673d50 msgr2=0x5569fc644ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:38.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.036+0000 7fdaa5000640 1 --2- 172.21.15.138:0/612385695 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x5569fc673d50 0x5569fc644ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:38.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.036+0000 7fdaa5000640 1 -- 172.21.15.138:0/612385695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fc695010 msgr2=0x5569fc66dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:38.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.036+0000 7fdaa5000640 1 --2- 172.21.15.138:0/612385695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fc695010 0x5569fc66dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:38.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.036+0000 7fdaa5000640 1 -- 172.21.15.138:0/612385695 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569fc670c00 con 0x5569fc64b190 2024-01-21T22:25:38.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.038+0000 7fdaa5000640 1 --2- 172.21.15.138:0/612385695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc64b190 0x5569fc644990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fda90019000 tx=0x7fda90014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de4086c4c3a13869 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:38.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.042+0000 7fda94a00640 1 -- 172.21.15.138:0/612385695 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90021040 con 0x5569fc64b190 2024-01-21T22:25:38.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.042+0000 7fda94a00640 1 -- 172.21.15.138:0/612385695 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda90022040 con 0x5569fc64b190 2024-01-21T22:25:38.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.042+0000 7fda94a00640 1 -- 172.21.15.138:0/612385695 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90003850 con 0x5569fc64b190 2024-01-21T22:25:38.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.043+0000 7fdaaa824740 1 -- 172.21.15.138:0/612385695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc64b190 msgr2=0x5569fc644990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:38.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.043+0000 7fdaaa824740 1 --2- 172.21.15.138:0/612385695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc64b190 0x5569fc644990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fda90019000 tx=0x7fda90014000 comp rx=0 tx=0).stop 2024-01-21T22:25:38.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.043+0000 7fdaaa824740 1 -- 172.21.15.138:0/612385695 shutdown_connections 2024-01-21T22:25:38.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.043+0000 7fdaaa824740 1 --2- 172.21.15.138:0/612385695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fc695010 0x5569fc66dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:38.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.043+0000 7fdaaa824740 1 --2- 172.21.15.138:0/612385695 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x5569fc673d50 0x5569fc644ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:38.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.043+0000 7fdaaa824740 1 --2- 172.21.15.138:0/612385695 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc64b190 0x5569fc644990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:38.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.043+0000 7fdaaa824740 1 -- 172.21.15.138:0/612385695 >> 172.21.15.138:0/612385695 conn(0x5569fc649bf0 msgr2=0x5569fc6410b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:38.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.044+0000 7fdaaa824740 1 -- 172.21.15.138:0/612385695 shutdown_connections 2024-01-21T22:25:38.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.044+0000 7fdaaa824740 1 -- 172.21.15.138:0/612385695 wait complete. 2024-01-21T22:25:38.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.045+0000 7fdaaa824740 1 -- start start 2024-01-21T22:25:38.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.046+0000 7fdaaa824740 1 Processor -- start 2024-01-21T22:25:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.048+0000 7fdaaa824740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fc64b190 0x5569fc687ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.048+0000 7fdaaa824740 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x5569fc673d50 0x5569fc6f0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.048+0000 7fdaaa824740 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc695010 0x5569fc6f0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.048+0000 7fdaaa824740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569fc687ff0 con 0x5569fc64b190 2024-01-21T22:25:38.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.048+0000 7fdaaa824740 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569fc670940 con 0x5569fc695010 2024-01-21T22:25:38.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.048+0000 7fdaaa824740 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569fc64a660 con 0x5569fc673d50 2024-01-21T22:25:38.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.048+0000 7fdaaa824740 10 client.0 _ll_register_callbacks cb 0x7fda8807aae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-01-21T22:25:38.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.049+0000 7fdaa5a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc695010 0x5569fc6f0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:38.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.049+0000 7fdaa5000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fc64b190 0x5569fc687ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:38.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.049+0000 7fda97c00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x5569fc673d50 0x5569fc6f0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:38.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.049+0000 7fdaa5a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc695010 0x5569fc6f0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:58362/0 (socket says 172.21.15.138:58362) 2024-01-21T22:25:38.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.049+0000 7fdaa5a00640 1 -- 172.21.15.138:0/1240507782 learned_addr learned my addr 172.21.15.138:0/1240507782 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:38.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.052+0000 7fdaa5a00640 1 -- 172.21.15.138:0/1240507782 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x5569fc673d50 msgr2=0x5569fc6f0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:38.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.052+0000 7fdaa5a00640 1 --2- 172.21.15.138:0/1240507782 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x5569fc673d50 0x5569fc6f0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:38.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.052+0000 7fdaa5a00640 1 -- 172.21.15.138:0/1240507782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fc64b190 msgr2=0x5569fc687ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:25:38.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.052+0000 7fdaa5a00640 1 --2- 172.21.15.138:0/1240507782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fc64b190 0x5569fc687ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:38.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.052+0000 7fdaa5a00640 1 -- 172.21.15.138:0/1240507782 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569fc566280 con 0x5569fc695010 2024-01-21T22:25:38.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.054+0000 7fdaa5a00640 1 --2- 172.21.15.138:0/1240507782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc695010 0x5569fc6f0b70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdaa00c5e20 tx=0x7fda98001250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faffc4606f6ff8fe server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:38.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.054+0000 7fda8e000640 10 client.0 ms_handle_connect on v2:172.21.15.152:3300/0 2024-01-21T22:25:38.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.058+0000 7fda8e000640 1 -- 172.21.15.138:0/1240507782 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98003b80 con 0x5569fc695010 2024-01-21T22:25:38.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.058+0000 7fda8e000640 1 -- 172.21.15.138:0/1240507782 <== mon.1 v2:172.21.15.152:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fda98003ce0 con 0x5569fc695010 2024-01-21T22:25:38.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.058+0000 7fdaaa824740 1 -- 172.21.15.138:0/1240507782 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda90014440 con 0x5569fc695010 2024-01-21T22:25:38.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.060+0000 7fda8e000640 1 -- 172.21.15.138:0/1240507782 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_get_version_reply(handle=1 version=10) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda98003ea0 con 0x5569fc695010 2024-01-21T22:25:38.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.060+0000 7fdaaa824740 10 client.4606 fetch_fsmap learned FSMap version 10 2024-01-21T22:25:38.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.060+0000 7fdaaa824740 1 -- 172.21.15.138:0/1240507782 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,fsmap=10,monmap=2+}) v3 -- 0x5569fc54bfd0 con 0x5569fc695010 2024-01-21T22:25:38.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.064+0000 7fda8e000640 1 -- 172.21.15.138:0/1240507782 <== mon.1 v2:172.21.15.152:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda90014440 con 0x5569fc695010 2024-01-21T22:25:38.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.064+0000 7fda8e000640 1 -- 172.21.15.138:0/1240507782 <== mon.1 v2:172.21.15.152:3300/0 5 ==== fsmap(e 10) v1 ==== 4444+0+0 (secure 0 0 0) 0x7fda98005110 con 0x5569fc695010 2024-01-21T22:25:38.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.065+0000 7fdaaa824740 10 client.4606 fetch_fsmap finished waiting for FSMap version 10 2024-01-21T22:25:38.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.065+0000 7fdaaa824740 10 client.4606 resolve_mds: resolved name 'd' to daemon mds.d 2024-01-21T22:25:38.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.065+0000 7fdaaa824740 20 client.4606 populate_metadata read hostname 'smithi138' 2024-01-21T22:25:38.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.065+0000 7fdaaa824740 1 --2- 172.21.15.138:0/1240507782 >> [v2:172.21.15.152:6834/2663179600,v1:172.21.15.152:6835/2663179600] conn(0x5569fc671f00 0x5569fc6722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:38.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.065+0000 7fdaaa824740 4 client.4606 mds_command: new command op to 4276 tid=0 multi_id=0 [{"prefix": "status"}] 2024-01-21T22:25:38.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.065+0000 7fdaaa824740 1 -- 172.21.15.138:0/1240507782 --> [v2:172.21.15.152:6834/2663179600,v1:172.21.15.152:6835/2663179600] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5569fc2b06c0 con 0x5569fc671f00 2024-01-21T22:25:38.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.065+0000 7fdaa5000640 1 --2- 172.21.15.138:0/1240507782 >> [v2:172.21.15.152:6834/2663179600,v1:172.21.15.152:6835/2663179600] conn(0x5569fc671f00 0x5569fc6722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:38.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.068+0000 7fdaa5000640 1 --2- 172.21.15.138:0/1240507782 >> [v2:172.21.15.152:6834/2663179600,v1:172.21.15.152:6835/2663179600] conn(0x5569fc671f00 0x5569fc6722e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c697562dd8db8dd8 server_cookie=cc56338b2d4e695 in_seq=0 out_seq=0 2024-01-21T22:25:38.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.068+0000 7fda8e000640 10 client.4606 ms_handle_connect on v2:172.21.15.152:6834/2663179600 2024-01-21T22:25:38.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.088+0000 7fda8e000640 1 -- 172.21.15.138:0/1240507782 <== mds.0 v2:172.21.15.152:6834/2663179600 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5569fc2b06c0 con 0x5569fc671f00 2024-01-21T22:25:38.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.088+0000 7fda8e000640 10 client.4606 handle_command_reply: tid=0 2024-01-21T22:25:38.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.088+0000 7fdaaa824740 1 client.4606 shutdown 2024-01-21T22:25:38.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.088+0000 7fdaaa824740 1 -- 172.21.15.138:0/1240507782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc695010 msgr2=0x5569fc6f0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:38.089 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.088+0000 7fdaaa824740 1 --2- 172.21.15.138:0/1240507782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc695010 0x5569fc6f0b70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdaa00c5e20 tx=0x7fda98001250 comp rx=0 tx=0).stop 2024-01-21T22:25:38.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.088+0000 7fdaaa824740 1 -- 172.21.15.138:0/1240507782 shutdown_connections 2024-01-21T22:25:38.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.088+0000 7fdaaa824740 1 --2- 172.21.15.138:0/1240507782 >> [v2:172.21.15.152:6834/2663179600,v1:172.21.15.152:6835/2663179600] conn(0x5569fc671f00 0x5569fc6722e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:38.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.088+0000 7fdaaa824740 1 --2- 172.21.15.138:0/1240507782 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x5569fc695010 0x5569fc6f0b70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:38.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.088+0000 7fdaaa824740 1 --2- 172.21.15.138:0/1240507782 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x5569fc673d50 0x5569fc6f0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:38.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.088+0000 7fdaaa824740 1 --2- 172.21.15.138:0/1240507782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5569fc64b190 0x5569fc687ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:38.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.088+0000 7fdaaa824740 1 -- 172.21.15.138:0/1240507782 >> 172.21.15.138:0/1240507782 conn(0x5569fc649bf0 msgr2=0x5569fc644990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:38.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.088+0000 7fda8e000640 0 client.4606 ms_handle_reset on v2:172.21.15.152:6834/2663179600 2024-01-21T22:25:38.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.089+0000 7fdaaa824740 1 -- 172.21.15.138:0/1240507782 shutdown_connections 2024-01-21T22:25:38.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.089+0000 7fdaaa824740 1 -- 172.21.15.138:0/1240507782 wait complete. 2024-01-21T22:25:38.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.089+0000 7fdaaa824740 20 client.4606 trim_cache size 0 max 16384 2024-01-21T22:25:38.091 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-01-21T22:25:38.091 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_fsid": "d488d446-149f-48d0-89ca-b2830ae72846", 2024-01-21T22:25:38.092 INFO:teuthology.orchestra.run.smithi138.stdout: "whoami": 0, 2024-01-21T22:25:38.092 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 4276, 2024-01-21T22:25:38.092 INFO:teuthology.orchestra.run.smithi138.stdout: "want_state": "up:active", 2024-01-21T22:25:38.092 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "up:active", 2024-01-21T22:25:38.092 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs", 2024-01-21T22:25:38.092 INFO:teuthology.orchestra.run.smithi138.stdout: "rank_uptime": 31.810951633999998, 2024-01-21T22:25:38.092 INFO:teuthology.orchestra.run.smithi138.stdout: "mdsmap_epoch": 10, 2024-01-21T22:25:38.092 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch": 25, 2024-01-21T22:25:38.092 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch_barrier": 23, 2024-01-21T22:25:38.092 INFO:teuthology.orchestra.run.smithi138.stdout: "uptime": 43.223997566999998 2024-01-21T22:25:38.092 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-01-21T22:25:38.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.090+0000 7fdaa7a00640 1 -- 172.21.15.138:0/22597936 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fda7806b380 msgr2=0x7fda7806d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:38.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.090+0000 7fdaa7a00640 1 --2- 172.21.15.138:0/22597936 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fda7806b380 0x7fda7806d840 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fda98007fd0 tx=0x7fda98009040 comp rx=0 tx=0).stop 2024-01-21T22:25:38.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.090+0000 7fdaa7a00640 1 -- 172.21.15.138:0/22597936 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdaa006c9b0 msgr2=0x7fdaa0156ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:38.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.091+0000 7fdaa7a00640 1 --2- 172.21.15.138:0/22597936 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdaa006c9b0 0x7fdaa0156ce0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fda8802bf50 tx=0x7fda88004730 comp rx=0 tx=0).stop 2024-01-21T22:25:38.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.091+0000 7fdaa7a00640 1 -- 172.21.15.138:0/22597936 shutdown_connections 2024-01-21T22:25:38.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.091+0000 7fdaa7a00640 1 --2- 172.21.15.138:0/22597936 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fda7806b380 0x7fda7806d840 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:38.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.091+0000 7fdaa7a00640 1 --2- 172.21.15.138:0/22597936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdaa00c5970 0x7fdaa01ef6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:38.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.091+0000 7fdaa7a00640 1 --2- 172.21.15.138:0/22597936 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdaa006c9b0 0x7fdaa0156ce0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:38.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.091+0000 7fdaa7a00640 1 --2- 172.21.15.138:0/22597936 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdaa006c5d0 0x7fdaa00ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:38.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.091+0000 7fdaa7a00640 1 -- 172.21.15.138:0/22597936 >> 172.21.15.138:0/22597936 conn(0x7fdaa00cb070 msgr2=0x7fdaa014fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:38.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.091+0000 7fdaa7a00640 1 -- 172.21.15.138:0/22597936 shutdown_connections 2024-01-21T22:25:38.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.091+0000 7fdaa7a00640 1 -- 172.21.15.138:0/22597936 wait complete. 2024-01-21T22:25:38.124 INFO:teuthology.run_tasks:Running task kclient... 2024-01-21T22:25:38.153 INFO:tasks.kclient:Mounting kernel clients... 2024-01-21T22:25:38.153 INFO:tasks.kclient:config is {'client.0': {}} 2024-01-21T22:25:38.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-21T22:25:38.153 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-01-21T22:25:38.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-21T22:25:38.211 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-01-21T22:25:38.310 INFO:teuthology.orchestra.run.smithi138.stderr:Cannot find device "ceph-brx" 2024-01-21T22:25:38.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-21T22:25:38.315 INFO:tasks.kclient:client.0 config is {} 2024-01-21T22:25:38.315 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-01-21T22:25:38.315 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-21T22:25:38.315 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-01-21T22:25:38.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.514+0000 7f6762c00640 1 Processor -- start 2024-01-21T22:25:38.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.514+0000 7f6762c00640 1 -- start start 2024-01-21T22:25:38.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.515+0000 7f6762c00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f675c1575a0 0x7f675c0c6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:38.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.515+0000 7f6762c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f675c0b0890 con 0x7f675c0c43f0 2024-01-21T22:25:38.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.515+0000 7f6762c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f675c0b0b70 con 0x7f675c06cca0 2024-01-21T22:25:38.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.515+0000 7f6762c00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f675c0c2870 con 0x7f675c1575a0 2024-01-21T22:25:38.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.516+0000 7f6760e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f675c0c43f0 0x7f675c06c590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:48190/0 (socket says 172.21.15.138:48190) 2024-01-21T22:25:38.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.516+0000 7f6760e00640 1 -- 172.21.15.138:0/4141156360 learned_addr learned my addr 172.21.15.138:0/4141156360 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:25:38.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.516+0000 7f6761800640 1 --2- 172.21.15.138:0/4141156360 >> v2:172.21.15.152:3300/0 conn(0x7f675c1575a0 0x7f675c0c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:38.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.519+0000 7f675be00640 1 -- 172.21.15.138:0/4141156360 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2525703541 0 0) 0x7f675c0b0890 con 0x7f675c0c43f0 2024-01-21T22:25:38.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.519+0000 7f675be00640 1 -- 172.21.15.138:0/4141156360 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f673c003520 con 0x7f675c0c43f0 2024-01-21T22:25:38.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.520+0000 7f6761800640 1 -- 172.21.15.138:0/4141156360 >> v1:172.21.15.152:6789/0 conn(0x7f675c06cca0 legacy=0x7f675c161180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:38.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.520+0000 7f6761800640 1 -- 172.21.15.138:0/4141156360 >> v1:172.21.15.138:6789/0 conn(0x7f675c0c43f0 legacy=0x7f675c06c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:38.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.520+0000 7f6761800640 1 -- 172.21.15.138:0/4141156360 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675c0b0e50 con 0x7f675c1575a0 2024-01-21T22:25:38.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.524+0000 7f6761800640 1 --2- 172.21.15.138:0/4141156360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f675c1575a0 0x7f675c0c6930 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f674c009d90 tx=0x7f674c02bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a294aad512671487 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:38.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.528+0000 7f675be00640 1 -- 172.21.15.138:0/4141156360 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c02ccf0 con 0x7f675c1575a0 2024-01-21T22:25:38.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.528+0000 7f675be00640 1 -- 172.21.15.138:0/4141156360 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f674c02ce50 con 0x7f675c1575a0 2024-01-21T22:25:38.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.528+0000 7f675be00640 1 -- 172.21.15.138:0/4141156360 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c034660 con 0x7f675c1575a0 2024-01-21T22:25:38.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.529+0000 7f6762c00640 1 -- 172.21.15.138:0/4141156360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f675c1575a0 msgr2=0x7f675c0c6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:38.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.529+0000 7f6762c00640 1 --2- 172.21.15.138:0/4141156360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f675c1575a0 0x7f675c0c6930 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f674c009d90 tx=0x7f674c02bf40 comp rx=0 tx=0).stop 2024-01-21T22:25:38.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.529+0000 7f6762c00640 1 -- 172.21.15.138:0/4141156360 shutdown_connections 2024-01-21T22:25:38.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.529+0000 7f6762c00640 1 --2- 172.21.15.138:0/4141156360 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f675c1575a0 0x7f675c0c6930 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:38.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.529+0000 7f6762c00640 1 -- 172.21.15.138:0/4141156360 >> 172.21.15.138:0/4141156360 conn(0x7f675c0c7920 msgr2=0x7f675c0c7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:38.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.529+0000 7f6762c00640 1 -- 172.21.15.138:0/4141156360 shutdown_connections 2024-01-21T22:25:38.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.529+0000 7f6762c00640 1 -- 172.21.15.138:0/4141156360 wait complete. 2024-01-21T22:25:38.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.532+0000 7f6762c00640 1 Processor -- start 2024-01-21T22:25:38.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.532+0000 7f6762c00640 1 -- start start 2024-01-21T22:25:38.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.533+0000 7f6762c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f675c06cca0 0x7f675c14bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:38.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.533+0000 7f6762c00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f675c0c43f0 0x7f675c14da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:38.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.533+0000 7f6762c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f675c1575a0 0x7f675c071540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:38.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.533+0000 7f6762c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c06c170 con 0x7f675c06cca0 2024-01-21T22:25:38.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.533+0000 7f6762c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675c154520 con 0x7f675c1575a0 2024-01-21T22:25:38.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.533+0000 7f6762c00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f675c14b690 con 0x7f675c0c43f0 2024-01-21T22:25:38.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.534+0000 7f6761800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f675c06cca0 0x7f675c14bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:38.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.534+0000 7f6760e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f675c0c43f0 0x7f675c14da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:38.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.534+0000 7f6761800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f675c06cca0 0x7f675c14bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55918/0 (socket says 172.21.15.138:55918) 2024-01-21T22:25:38.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.534+0000 7f6761800640 1 -- 172.21.15.138:0/1889063733 learned_addr learned my addr 172.21.15.138:0/1889063733 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:25:38.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.534+0000 7f6762200640 1 --2- 172.21.15.138:0/1889063733 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f675c1575a0 0x7f675c071540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:38.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.538+0000 7f6762200640 1 -- 172.21.15.138:0/1889063733 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f675c0c43f0 msgr2=0x7f675c14da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:38.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.538+0000 7f6762200640 1 --2- 172.21.15.138:0/1889063733 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f675c0c43f0 0x7f675c14da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:38.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.538+0000 7f6762200640 1 -- 172.21.15.138:0/1889063733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f675c06cca0 msgr2=0x7f675c14bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:38.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.538+0000 7f6762200640 1 --2- 172.21.15.138:0/1889063733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f675c06cca0 0x7f675c14bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:38.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.538+0000 7f6762200640 1 -- 172.21.15.138:0/1889063733 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674c009a70 con 0x7f675c1575a0 2024-01-21T22:25:38.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.540+0000 7f6762200640 1 --2- 172.21.15.138:0/1889063733 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f675c1575a0 0x7f675c071540 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f675400c3f0 tx=0x7f675400c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:38.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.543+0000 7f6759600640 1 -- 172.21.15.138:0/1889063733 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67540052d0 con 0x7f675c1575a0 2024-01-21T22:25:38.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.543+0000 7f6759600640 1 -- 172.21.15.138:0/1889063733 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6754005430 con 0x7f675c1575a0 2024-01-21T22:25:38.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.543+0000 7f6759600640 1 -- 172.21.15.138:0/1889063733 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6754007260 con 0x7f675c1575a0 2024-01-21T22:25:38.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.543+0000 7f6762c00640 1 -- 172.21.15.138:0/1889063733 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f675c14e420 con 0x7f675c1575a0 2024-01-21T22:25:38.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.544+0000 7f6762c00640 1 -- 172.21.15.138:0/1889063733 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f675c14c0b0 con 0x7f675c1575a0 2024-01-21T22:25:38.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.545+0000 7f6762c00640 1 -- 172.21.15.138:0/1889063733 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f675c04ea30 con 0x7f675c1575a0 2024-01-21T22:25:38.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.791+0000 7f6759600640 1 -- 172.21.15.138:0/1889063733 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f67540073c0 con 0x7f675c1575a0 2024-01-21T22:25:38.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.793+0000 7f6759600640 1 --2- 172.21.15.138:0/1889063733 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f673406b380 0x7f673406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:25:38.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.793+0000 7f6759600640 1 -- 172.21.15.138:0/1889063733 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f6754089340 con 0x7f675c1575a0 2024-01-21T22:25:38.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.794+0000 7f6761800640 1 --2- 172.21.15.138:0/1889063733 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f673406b380 0x7f673406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:25:38.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.795+0000 7f6761800640 1 --2- 172.21.15.138:0/1889063733 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f673406b380 0x7f673406d840 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f674c02bf80 tx=0x7f674c030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:25:38.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:38.804+0000 7f6759600640 1 -- 172.21.15.138:0/1889063733 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f67540541d0 con 0x7f675c1575a0 2024-01-21T22:25:39.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:39.050+0000 7f6762c00640 1 -- 172.21.15.138:0/1889063733 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f675c14c0b0 con 0x7f675c1575a0 2024-01-21T22:25:39.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:39.056+0000 7f6759600640 1 -- 172.21.15.138:0/1889063733 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v10) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6754020060 con 0x7f675c1575a0 2024-01-21T22:25:39.057 INFO:teuthology.orchestra.run.smithi138.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-01-21T22:25:39.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:39.059+0000 7f6762c00640 1 -- 172.21.15.138:0/1889063733 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f673406b380 msgr2=0x7f673406d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:39.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:39.059+0000 7f6762c00640 1 --2- 172.21.15.138:0/1889063733 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f673406b380 0x7f673406d840 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f674c02bf80 tx=0x7f674c030000 comp rx=0 tx=0).stop 2024-01-21T22:25:39.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:39.059+0000 7f6762c00640 1 -- 172.21.15.138:0/1889063733 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f675c1575a0 msgr2=0x7f675c071540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:25:39.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:39.059+0000 7f6762c00640 1 --2- 172.21.15.138:0/1889063733 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f675c1575a0 0x7f675c071540 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f675400c3f0 tx=0x7f675400c8b0 comp rx=0 tx=0).stop 2024-01-21T22:25:39.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:39.059+0000 7f6762c00640 1 -- 172.21.15.138:0/1889063733 shutdown_connections 2024-01-21T22:25:39.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:39.059+0000 7f6762c00640 1 --2- 172.21.15.138:0/1889063733 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f673406b380 0x7f673406d840 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:39.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:39.059+0000 7f6762c00640 1 --2- 172.21.15.138:0/1889063733 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f675c1575a0 0x7f675c071540 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:39.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:39.059+0000 7f6762c00640 1 --2- 172.21.15.138:0/1889063733 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f675c0c43f0 0x7f675c14da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:39.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:39.059+0000 7f6762c00640 1 --2- 172.21.15.138:0/1889063733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f675c06cca0 0x7f675c14bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:25:39.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:39.059+0000 7f6762c00640 1 -- 172.21.15.138:0/1889063733 >> 172.21.15.138:0/1889063733 conn(0x7f675c0c7920 msgr2=0x7f675c06c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:25:39.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:39.059+0000 7f6762c00640 1 -- 172.21.15.138:0/1889063733 shutdown_connections 2024-01-21T22:25:39.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:25:39.060+0000 7f6762c00640 1 -- 172.21.15.138:0/1889063733 wait complete. 2024-01-21T22:25:39.079 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-01-21T22:25:39.079 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-01-21T22:25:39.079 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-01-21T22:25:39.079 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-01-21T22:25:39.079 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-01-21T22:25:39.079 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-01-21T22:25:39.080 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-01-21T22:25:39.080 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-01-21T22:25:39.080 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-01-21T22:25:39.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-21T22:25:39.080 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-01-21T22:25:39.130 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-01-21T22:25:39.130 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-01-21T22:25:39.130 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-01-21T22:25:39.130 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-01-21T22:25:39.130 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-01-21T22:25:39.130 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-01-21T22:25:39.130 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-21T22:25:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-01-21T22:25:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-01-21T22:25:39.131 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-21T22:25:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-01-21T22:25:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-01-21T22:25:39.131 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-01-21T22:25:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-01-21T22:25:39.131 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-01-21T22:25:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-01-21T22:25:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-01-21T22:25:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42210sec preferred_lft 42210sec 2024-01-21T22:25:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link 2024-01-21T22:25:39.131 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-01-21T22:25:39.133 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-01-21T22:25:39.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-21T22:25:39.133 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2024-01-21T22:25:39.133 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-01-21T22:25:39.133 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add name ceph-brx type bridge 2024-01-21T22:25:39.133 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr flush dev ceph-brx 2024-01-21T22:25:39.133 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set ceph-brx up 2024-01-21T22:25:39.134 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-01-21T22:25:39.134 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-01-21T22:25:39.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-21T22:25:39.571 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-01-21T22:25:39.649 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-01-21T22:25:39.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-21T22:25:39.652 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-01-21T22:25:39.710 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-01-21T22:25:39.711 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-01-21T22:25:39.711 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-01-21T22:25:39.711 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-01-21T22:25:39.711 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-01-21T22:25:39.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-21T22:25:39.711 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2024-01-21T22:25:39.712 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-01-21T22:25:39.712 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-01-21T22:25:39.712 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-01-21T22:25:39.712 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-01-21T22:25:39.712 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-01-21T22:25:40.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-21T22:25:40.521 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-01-21T22:25:40.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-21T22:25:40.574 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list-id 2024-01-21T22:25:40.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-21T22:25:40.623 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2024-01-21T22:25:40.623 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-01-21T22:25:40.623 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-21T22:25:40.623 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-01-21T22:25:40.624 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-01-21T22:25:40.762 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-01-21T22:25:40.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-21T22:25:40.763 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2024-01-21T22:25:40.763 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-01-21T22:25:40.763 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-01-21T22:25:40.763 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-01-21T22:25:40.763 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-01-21T22:25:40.763 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-01-21T22:25:40.763 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-01-21T22:25:40.763 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-01-21T22:25:41.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-21T22:25:41.084 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec bash -c ' 2024-01-21T22:25:41.085 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-01-21T22:25:41.085 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2024-01-21T22:25:41.085 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set dev brx.0 master ceph-brx 2024-01-21T22:25:41.085 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-01-21T22:25:41.241 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-21T22:25:41.241 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-01-21T22:25:41.284 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-01-21T22:25:41.284 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-01-21T22:25:41.284 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-01-21T22:25:41.377 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-01-21T22:25:41.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-21T22:25:41.377 DEBUG:teuthology.orchestra.run.smithi138:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-01-21T22:25:41.551 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-01-21T22:25:41.551 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-01-21T22:25:42.484 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-01-21T22:25:42.484 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-01-21T22:25:42.484 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.138:3300,172.21.15.152:3300,172.21.15.152:3301" 2024-01-21T22:25:42.484 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: trying mount with new device syntax: 0@d488d446-149f-48d0-89ca-b2830ae72846.cephfs=/ 2024-01-21T22:25:42.484 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.138:3300/172.21.15.152:3300/172.21.15.152:3301" will pass to kernel 2024-01-21T22:25:42.489 INFO:teuthology.orchestra.run.smithi138.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-01-21T22:25:42.489 INFO:teuthology.orchestra.run.smithi138.stdout: You just mounted a file system that supports labels which does not 2024-01-21T22:25:42.489 INFO:teuthology.orchestra.run.smithi138.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-01-21T22:25:42.489 INFO:teuthology.orchestra.run.smithi138.stdout: applications will generate AVC messages and not be allowed access to 2024-01-21T22:25:42.489 INFO:teuthology.orchestra.run.smithi138.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-01-21T22:25:42.496 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-01-21T22:25:42.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-21T22:25:42.496 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-01-21T22:25:42.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-21T22:25:42.672 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-21T22:25:42.923 INFO:teuthology.orchestra.run.smithi138.stdout:client4496 2024-01-21T22:25:42.924 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:25:42.924 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-21T22:25:42.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-21T22:25:42.979 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-21T22:25:43.205 INFO:teuthology.orchestra.run.smithi138.stdout:client4496 2024-01-21T22:25:43.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-21T22:25:43.205 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-01-21T22:25:43.447 INFO:teuthology.orchestra.run.smithi138.stdout:client4496 2024-01-21T22:25:43.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-21T22:25:43.448 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d488d446-149f-48d0-89ca-b2830ae72846.client4496/status') 2024-01-21T22:25:43.673 INFO:teuthology.orchestra.run.smithi138.stdout:instance: client.4496 (3)172.21.15.138:0/928078195 2024-01-21T22:25:43.673 INFO:teuthology.orchestra.run.smithi138.stdout:blocklisted: false 2024-01-21T22:25:43.674 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records in 2024-01-21T22:25:43.674 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records out 2024-01-21T22:25:43.674 INFO:teuthology.orchestra.run.smithi138.stderr:70 bytes copied, 8.0317e-05 s, 872 kB/s 2024-01-21T22:25:43.681 INFO:teuthology.run_tasks:Running task workunit... 2024-01-21T22:25:43.693 INFO:tasks.workunit:Pulling workunits from ref eb1b049f9003ac37e0a3ab33496ad1924d301254 2024-01-21T22:25:43.693 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-01-21T22:25:43.693 INFO:tasks.workunit:timeout=3h 2024-01-21T22:25:43.693 INFO:tasks.workunit:cleanup=True 2024-01-21T22:25:43.694 DEBUG:teuthology.orchestra.run.smithi138:> stat -- /home/ubuntu/cephtest/mnt.0 2024-01-21T22:25:43.755 INFO:teuthology.orchestra.run.smithi138.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-01-21T22:25:43.755 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-01-21T22:25:43.755 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 27h/39d Inode: 1 Links: 2 2024-01-21T22:25:43.755 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-01-21T22:25:43.755 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-01-21T22:25:43.755 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-01-21T22:25:43.755 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-01-21 22:25:06.578665207 +0000 2024-01-21T22:25:43.755 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-01-21 22:25:42.587244329 +0000 2024-01-21T22:25:43.756 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-01-21 22:25:06.578665207 +0000 2024-01-21T22:25:43.756 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-01-21T22:25:43.756 DEBUG:teuthology.orchestra.run.smithi138:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-01-21T22:25:44.200 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout eb1b049f9003ac37e0a3ab33496ad1924d301254 2024-01-21T22:25:44.260 INFO:tasks.workunit.client.0.smithi138.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-01-21T22:26:04.808 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:26:04.817 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:26:34.940 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:26:34.949 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:27:00.799 INFO:tasks.workunit.client.0.smithi138.stderr:Updating files: 64% (7846/12180) Updating files: 65% (7917/12180) Updating files: 66% (8039/12180) Updating files: 67% (8161/12180) Updating files: 68% (8283/12180) Updating files: 69% (8405/12180) Updating files: 70% (8526/12180) Updating files: 71% (8648/12180) Updating files: 72% (8770/12180) Updating files: 73% (8892/12180) Updating files: 74% (9014/12180) Updating files: 75% (9135/12180) Updating files: 76% (9257/12180) Updating files: 77% (9379/12180) Updating files: 78% (9501/12180) Updating files: 79% (9623/12180) Updating files: 80% (9744/12180) Updating files: 81% (9866/12180) Updating files: 82% (9988/12180) Updating files: 83% (10110/12180) Updating files: 84% (10232/12180) Updating files: 85% (10353/12180) Updating files: 86% (10475/12180) Updating files: 87% (10597/12180) Updating files: 88% (10719/12180) Updating files: 89% (10841/12180) Updating files: 90% (10962/12180) Updating files: 91% (11084/12180) Updating files: 92% (11206/12180) Updating files: 93% (11328/12180) Updating files: 94% (11450/12180) Updating files: 95% (11571/12180) Updating files: 96% (11693/12180) Updating files: 97% (11815/12180) Updating files: 98% (11937/12180) Updating files: 99% (12059/12180) Updating files: 100% (12180/12180) Updating files: 100% (12180/12180), done. 2024-01-21T22:27:01.354 INFO:tasks.workunit.client.0.smithi138.stderr:Note: switching to 'eb1b049f9003ac37e0a3ab33496ad1924d301254'. 2024-01-21T22:27:01.354 INFO:tasks.workunit.client.0.smithi138.stderr: 2024-01-21T22:27:01.354 INFO:tasks.workunit.client.0.smithi138.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-01-21T22:27:01.354 INFO:tasks.workunit.client.0.smithi138.stderr:changes and commit them, and you can discard any commits you make in this 2024-01-21T22:27:01.354 INFO:tasks.workunit.client.0.smithi138.stderr:state without impacting any branches by switching back to a branch. 2024-01-21T22:27:01.354 INFO:tasks.workunit.client.0.smithi138.stderr: 2024-01-21T22:27:01.354 INFO:tasks.workunit.client.0.smithi138.stderr:If you want to create a new branch to retain commits you create, you may 2024-01-21T22:27:01.354 INFO:tasks.workunit.client.0.smithi138.stderr:do so (now or later) by using -c with the switch command. Example: 2024-01-21T22:27:01.354 INFO:tasks.workunit.client.0.smithi138.stderr: 2024-01-21T22:27:01.354 INFO:tasks.workunit.client.0.smithi138.stderr: git switch -c 2024-01-21T22:27:01.354 INFO:tasks.workunit.client.0.smithi138.stderr: 2024-01-21T22:27:01.354 INFO:tasks.workunit.client.0.smithi138.stderr:Or undo this operation with: 2024-01-21T22:27:01.354 INFO:tasks.workunit.client.0.smithi138.stderr: 2024-01-21T22:27:01.355 INFO:tasks.workunit.client.0.smithi138.stderr: git switch - 2024-01-21T22:27:01.355 INFO:tasks.workunit.client.0.smithi138.stderr: 2024-01-21T22:27:01.355 INFO:tasks.workunit.client.0.smithi138.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-01-21T22:27:01.355 INFO:tasks.workunit.client.0.smithi138.stderr: 2024-01-21T22:27:01.355 INFO:tasks.workunit.client.0.smithi138.stderr:HEAD is now at eb1b049f900 Merge PR #52652 into wip-vshankar-testing-20240119.075157 2024-01-21T22:27:01.357 DEBUG:teuthology.orchestra.run.smithi138:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-01-21T22:27:01.415 INFO:tasks.workunit.client.0.smithi138.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-01-21T22:27:01.422 INFO:tasks.workunit.client.0.smithi138.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-01-21T22:27:01.422 INFO:tasks.workunit.client.0.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-01-21T22:27:02.219 INFO:tasks.workunit.client.0.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-01-21T22:27:02.340 INFO:tasks.workunit.client.0.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-01-21T22:27:02.426 INFO:tasks.workunit.client.0.smithi138.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-01-21T22:27:02.433 INFO:tasks.workunit.client.0.smithi138.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-01-21T22:27:02.433 INFO:tasks.workunit.client.0.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-01-21T22:27:02.515 INFO:tasks.workunit.client.0.smithi138.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-01-21T22:27:02.524 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:27:02.524 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-01-21T22:27:02.572 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-01-21T22:27:02.573 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-01-21T22:27:02.574 DEBUG:teuthology.orchestra.run.smithi138:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=eb1b049f9003ac37e0a3ab33496ad1924d301254 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-01-21T22:27:02.775 INFO:tasks.workunit.client.0.smithi138.stderr:+ dbench 1 2024-01-21T22:27:02.793 INFO:tasks.workunit.client.0.smithi138.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-01-21T22:27:02.794 INFO:tasks.workunit.client.0.smithi138.stdout: 2024-01-21T22:27:02.794 INFO:tasks.workunit.client.0.smithi138.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-01-21T22:27:02.803 INFO:tasks.workunit.client.0.smithi138.stdout:0 of 1 processes prepared for launch 0 sec 2024-01-21T22:27:02.904 INFO:tasks.workunit.client.0.smithi138.stdout:1 of 1 processes prepared for launch 0 sec 2024-01-21T22:27:02.904 INFO:tasks.workunit.client.0.smithi138.stdout:releasing clients 2024-01-21T22:27:03.905 INFO:tasks.workunit.client.0.smithi138.stdout: 1 147 6.28 MB/sec warmup 1 sec latency 96.167 ms 2024-01-21T22:27:04.904 INFO:tasks.workunit.client.0.smithi138.stdout: 1 255 4.83 MB/sec warmup 2 sec latency 98.959 ms 2024-01-21T22:27:05.062 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:27:05.070 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:27:05.905 INFO:tasks.workunit.client.0.smithi138.stdout: 1 357 4.26 MB/sec warmup 3 sec latency 100.774 ms 2024-01-21T22:27:06.905 INFO:tasks.workunit.client.0.smithi138.stdout: 1 453 3.98 MB/sec warmup 4 sec latency 121.939 ms 2024-01-21T22:27:07.905 INFO:tasks.workunit.client.0.smithi138.stdout: 1 646 4.21 MB/sec warmup 5 sec latency 104.634 ms 2024-01-21T22:27:08.905 INFO:tasks.workunit.client.0.smithi138.stdout: 1 710 3.53 MB/sec warmup 6 sec latency 400.030 ms 2024-01-21T22:27:09.905 INFO:tasks.workunit.client.0.smithi138.stdout: 1 711 3.03 MB/sec warmup 7 sec latency 1074.784 ms 2024-01-21T22:27:10.905 INFO:tasks.workunit.client.0.smithi138.stdout: 1 811 2.68 MB/sec warmup 8 sec latency 401.045 ms 2024-01-21T22:27:11.905 INFO:tasks.workunit.client.0.smithi138.stdout: 1 947 2.42 MB/sec warmup 9 sec latency 138.974 ms 2024-01-21T22:27:12.905 INFO:tasks.workunit.client.0.smithi138.stdout: 1 1095 2.29 MB/sec warmup 10 sec latency 165.829 ms 2024-01-21T22:27:13.906 INFO:tasks.workunit.client.0.smithi138.stdout: 1 1244 2.10 MB/sec warmup 11 sec latency 98.784 ms 2024-01-21T22:27:14.906 INFO:tasks.workunit.client.0.smithi138.stdout: 1 1481 2.17 MB/sec warmup 12 sec latency 130.974 ms 2024-01-21T22:27:15.906 INFO:tasks.workunit.client.0.smithi138.stdout: 1 1662 2.02 MB/sec warmup 13 sec latency 92.276 ms 2024-01-21T22:27:16.906 INFO:tasks.workunit.client.0.smithi138.stdout: 1 1909 1.89 MB/sec warmup 14 sec latency 139.626 ms 2024-01-21T22:27:17.906 INFO:tasks.workunit.client.0.smithi138.stdout: 1 2294 1.85 MB/sec warmup 15 sec latency 181.889 ms 2024-01-21T22:27:18.906 INFO:tasks.workunit.client.0.smithi138.stdout: 1 2423 1.76 MB/sec warmup 16 sec latency 131.250 ms 2024-01-21T22:27:19.907 INFO:tasks.workunit.client.0.smithi138.stdout: 1 2558 1.70 MB/sec warmup 17 sec latency 118.483 ms 2024-01-21T22:27:20.907 INFO:tasks.workunit.client.0.smithi138.stdout: 1 2761 1.78 MB/sec warmup 18 sec latency 256.327 ms 2024-01-21T22:27:21.907 INFO:tasks.workunit.client.0.smithi138.stdout: 1 2922 1.76 MB/sec warmup 19 sec latency 105.128 ms 2024-01-21T22:27:22.907 INFO:tasks.workunit.client.0.smithi138.stdout: 1 3012 1.68 MB/sec warmup 20 sec latency 220.350 ms 2024-01-21T22:27:23.908 INFO:tasks.workunit.client.0.smithi138.stdout: 1 3270 1.68 MB/sec warmup 21 sec latency 112.313 ms 2024-01-21T22:27:24.907 INFO:tasks.workunit.client.0.smithi138.stdout: 1 3494 1.74 MB/sec warmup 22 sec latency 104.867 ms 2024-01-21T22:27:25.907 INFO:tasks.workunit.client.0.smithi138.stdout: 1 3617 1.68 MB/sec warmup 23 sec latency 189.318 ms 2024-01-21T22:27:26.907 INFO:tasks.workunit.client.0.smithi138.stdout: 1 3715 1.62 MB/sec warmup 24 sec latency 84.705 ms 2024-01-21T22:27:27.908 INFO:tasks.workunit.client.0.smithi138.stdout: 1 3795 1.59 MB/sec warmup 25 sec latency 101.460 ms 2024-01-21T22:27:28.908 INFO:tasks.workunit.client.0.smithi138.stdout: 1 3828 1.53 MB/sec warmup 26 sec latency 178.273 ms 2024-01-21T22:27:29.908 INFO:tasks.workunit.client.0.smithi138.stdout: 1 3885 1.47 MB/sec warmup 27 sec latency 79.208 ms 2024-01-21T22:27:30.908 INFO:tasks.workunit.client.0.smithi138.stdout: 1 3932 1.43 MB/sec warmup 28 sec latency 465.793 ms 2024-01-21T22:27:31.908 INFO:tasks.workunit.client.0.smithi138.stdout: 1 3939 1.38 MB/sec warmup 29 sec latency 1191.249 ms 2024-01-21T22:27:32.908 INFO:tasks.workunit.client.0.smithi138.stdout: 1 3959 1.34 MB/sec warmup 30 sec latency 517.250 ms 2024-01-21T22:27:33.908 INFO:tasks.workunit.client.0.smithi138.stdout: 1 4005 1.29 MB/sec warmup 31 sec latency 191.840 ms 2024-01-21T22:27:34.908 INFO:tasks.workunit.client.0.smithi138.stdout: 1 4100 1.25 MB/sec warmup 32 sec latency 119.657 ms 2024-01-21T22:27:35.198 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:27:35.207 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:27:35.909 INFO:tasks.workunit.client.0.smithi138.stdout: 1 4129 1.21 MB/sec warmup 33 sec latency 194.948 ms 2024-01-21T22:27:36.909 INFO:tasks.workunit.client.0.smithi138.stdout: 1 4168 1.18 MB/sec warmup 34 sec latency 294.355 ms 2024-01-21T22:27:37.909 INFO:tasks.workunit.client.0.smithi138.stdout: 1 4283 1.15 MB/sec warmup 35 sec latency 178.267 ms 2024-01-21T22:27:38.909 INFO:tasks.workunit.client.0.smithi138.stdout: 1 4382 1.13 MB/sec warmup 36 sec latency 139.202 ms 2024-01-21T22:27:39.909 INFO:tasks.workunit.client.0.smithi138.stdout: 1 4493 1.11 MB/sec warmup 37 sec latency 186.991 ms 2024-01-21T22:27:40.909 INFO:tasks.workunit.client.0.smithi138.stdout: 1 4628 1.11 MB/sec warmup 38 sec latency 122.121 ms 2024-01-21T22:27:41.910 INFO:tasks.workunit.client.0.smithi138.stdout: 1 4743 1.08 MB/sec warmup 39 sec latency 115.906 ms 2024-01-21T22:27:42.909 INFO:tasks.workunit.client.0.smithi138.stdout: 1 4959 1.13 MB/sec warmup 40 sec latency 128.350 ms 2024-01-21T22:27:43.910 INFO:tasks.workunit.client.0.smithi138.stdout: 1 5082 1.10 MB/sec warmup 41 sec latency 171.646 ms 2024-01-21T22:27:44.910 INFO:tasks.workunit.client.0.smithi138.stdout: 1 5208 1.08 MB/sec warmup 42 sec latency 282.438 ms 2024-01-21T22:27:45.910 INFO:tasks.workunit.client.0.smithi138.stdout: 1 5367 1.06 MB/sec warmup 43 sec latency 142.214 ms 2024-01-21T22:27:46.910 INFO:tasks.workunit.client.0.smithi138.stdout: 1 5736 1.04 MB/sec warmup 44 sec latency 134.927 ms 2024-01-21T22:27:47.911 INFO:tasks.workunit.client.0.smithi138.stdout: 1 5865 1.04 MB/sec warmup 45 sec latency 386.077 ms 2024-01-21T22:27:48.910 INFO:tasks.workunit.client.0.smithi138.stdout: 1 5960 1.02 MB/sec warmup 46 sec latency 506.532 ms 2024-01-21T22:27:49.911 INFO:tasks.workunit.client.0.smithi138.stdout: 1 6044 1.02 MB/sec warmup 47 sec latency 131.500 ms 2024-01-21T22:27:50.911 INFO:tasks.workunit.client.0.smithi138.stdout: 1 6154 1.00 MB/sec warmup 48 sec latency 147.736 ms 2024-01-21T22:27:51.911 INFO:tasks.workunit.client.0.smithi138.stdout: 1 6451 1.06 MB/sec warmup 49 sec latency 123.401 ms 2024-01-21T22:27:52.911 INFO:tasks.workunit.client.0.smithi138.stdout: 1 6555 1.05 MB/sec warmup 50 sec latency 136.024 ms 2024-01-21T22:27:53.911 INFO:tasks.workunit.client.0.smithi138.stdout: 1 6779 1.05 MB/sec warmup 51 sec latency 116.706 ms 2024-01-21T22:27:54.911 INFO:tasks.workunit.client.0.smithi138.stdout: 1 7046 1.10 MB/sec warmup 52 sec latency 120.882 ms 2024-01-21T22:27:55.912 INFO:tasks.workunit.client.0.smithi138.stdout: 1 7173 1.09 MB/sec warmup 53 sec latency 142.144 ms 2024-01-21T22:27:56.911 INFO:tasks.workunit.client.0.smithi138.stdout: 1 7261 1.07 MB/sec warmup 54 sec latency 87.446 ms 2024-01-21T22:27:57.911 INFO:tasks.workunit.client.0.smithi138.stdout: 1 7341 1.07 MB/sec warmup 55 sec latency 107.954 ms 2024-01-21T22:27:58.911 INFO:tasks.workunit.client.0.smithi138.stdout: 1 7373 1.05 MB/sec warmup 56 sec latency 208.802 ms 2024-01-21T22:27:59.912 INFO:tasks.workunit.client.0.smithi138.stdout: 1 7392 1.03 MB/sec warmup 57 sec latency 351.559 ms 2024-01-21T22:28:00.912 INFO:tasks.workunit.client.0.smithi138.stdout: 1 7441 1.02 MB/sec warmup 58 sec latency 202.775 ms 2024-01-21T22:28:01.912 INFO:tasks.workunit.client.0.smithi138.stdout: 1 7492 1.00 MB/sec warmup 59 sec latency 119.442 ms 2024-01-21T22:28:02.912 INFO:tasks.workunit.client.0.smithi138.stdout: 1 7543 0.99 MB/sec warmup 60 sec latency 116.017 ms 2024-01-21T22:28:03.912 INFO:tasks.workunit.client.0.smithi138.stdout: 1 7627 0.97 MB/sec warmup 61 sec latency 98.119 ms 2024-01-21T22:28:04.912 INFO:tasks.workunit.client.0.smithi138.stdout: 1 7668 0.95 MB/sec warmup 62 sec latency 176.036 ms 2024-01-21T22:28:05.365 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:28:05.373 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:28:05.912 INFO:tasks.workunit.client.0.smithi138.stdout: 1 7701 0.94 MB/sec warmup 63 sec latency 370.999 ms 2024-01-21T22:28:06.913 INFO:tasks.workunit.client.0.smithi138.stdout: 1 7812 0.93 MB/sec warmup 64 sec latency 170.769 ms 2024-01-21T22:28:07.913 INFO:tasks.workunit.client.0.smithi138.stdout: 1 7921 0.92 MB/sec warmup 65 sec latency 182.324 ms 2024-01-21T22:28:08.913 INFO:tasks.workunit.client.0.smithi138.stdout: 1 8025 0.91 MB/sec warmup 66 sec latency 122.915 ms 2024-01-21T22:28:09.913 INFO:tasks.workunit.client.0.smithi138.stdout: 1 8160 0.91 MB/sec warmup 67 sec latency 132.741 ms 2024-01-21T22:28:10.913 INFO:tasks.workunit.client.0.smithi138.stdout: 1 8243 0.90 MB/sec warmup 68 sec latency 127.929 ms 2024-01-21T22:28:11.913 INFO:tasks.workunit.client.0.smithi138.stdout: 1 8360 0.89 MB/sec warmup 69 sec latency 87.744 ms 2024-01-21T22:28:12.913 INFO:tasks.workunit.client.0.smithi138.stdout: 1 8555 0.92 MB/sec warmup 70 sec latency 169.883 ms 2024-01-21T22:28:13.913 INFO:tasks.workunit.client.0.smithi138.stdout: 1 8751 0.91 MB/sec warmup 71 sec latency 196.749 ms 2024-01-21T22:28:14.914 INFO:tasks.workunit.client.0.smithi138.stdout: 1 8963 0.90 MB/sec warmup 72 sec latency 132.039 ms 2024-01-21T22:28:15.914 INFO:tasks.workunit.client.0.smithi138.stdout: 1 9352 0.89 MB/sec warmup 73 sec latency 136.676 ms 2024-01-21T22:28:16.914 INFO:tasks.workunit.client.0.smithi138.stdout: 1 9498 0.89 MB/sec warmup 74 sec latency 120.369 ms 2024-01-21T22:28:17.914 INFO:tasks.workunit.client.0.smithi138.stdout: 1 9570 0.89 MB/sec warmup 75 sec latency 118.193 ms 2024-01-21T22:28:18.914 INFO:tasks.workunit.client.0.smithi138.stdout: 1 9684 0.88 MB/sec warmup 76 sec latency 149.503 ms 2024-01-21T22:28:19.914 INFO:tasks.workunit.client.0.smithi138.stdout: 1 9979 0.91 MB/sec warmup 77 sec latency 125.981 ms 2024-01-21T22:28:20.914 INFO:tasks.workunit.client.0.smithi138.stdout: 1 10083 0.92 MB/sec warmup 78 sec latency 129.471 ms 2024-01-21T22:28:21.914 INFO:tasks.workunit.client.0.smithi138.stdout: 1 10276 0.92 MB/sec warmup 79 sec latency 180.808 ms 2024-01-21T22:28:22.915 INFO:tasks.workunit.client.0.smithi138.stdout: 1 10288 0.91 MB/sec warmup 80 sec latency 907.254 ms 2024-01-21T22:28:23.915 INFO:tasks.workunit.client.0.smithi138.stdout: 1 10395 0.91 MB/sec warmup 81 sec latency 415.188 ms 2024-01-21T22:28:24.915 INFO:tasks.workunit.client.0.smithi138.stdout: 1 10604 0.93 MB/sec warmup 82 sec latency 108.134 ms 2024-01-21T22:28:25.915 INFO:tasks.workunit.client.0.smithi138.stdout: 1 10741 0.93 MB/sec warmup 83 sec latency 105.734 ms 2024-01-21T22:28:26.915 INFO:tasks.workunit.client.0.smithi138.stdout: 1 10863 0.93 MB/sec warmup 84 sec latency 92.822 ms 2024-01-21T22:28:27.915 INFO:tasks.workunit.client.0.smithi138.stdout: 1 10901 0.92 MB/sec warmup 85 sec latency 114.473 ms 2024-01-21T22:28:28.915 INFO:tasks.workunit.client.0.smithi138.stdout: 1 10950 0.91 MB/sec warmup 86 sec latency 101.964 ms 2024-01-21T22:28:29.916 INFO:tasks.workunit.client.0.smithi138.stdout: 1 11009 0.90 MB/sec warmup 87 sec latency 196.970 ms 2024-01-21T22:28:30.916 INFO:tasks.workunit.client.0.smithi138.stdout: 1 11042 0.89 MB/sec warmup 88 sec latency 146.894 ms 2024-01-21T22:28:31.916 INFO:tasks.workunit.client.0.smithi138.stdout: 1 11100 0.88 MB/sec warmup 89 sec latency 80.567 ms 2024-01-21T22:28:32.916 INFO:tasks.workunit.client.0.smithi138.stdout: 1 11183 0.87 MB/sec warmup 90 sec latency 138.865 ms 2024-01-21T22:28:33.916 INFO:tasks.workunit.client.0.smithi138.stdout: 1 11213 0.86 MB/sec warmup 91 sec latency 116.976 ms 2024-01-21T22:28:34.916 INFO:tasks.workunit.client.0.smithi138.stdout: 1 11244 0.85 MB/sec warmup 92 sec latency 255.404 ms 2024-01-21T22:28:35.505 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:28:35.513 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:28:35.917 INFO:tasks.workunit.client.0.smithi138.stdout: 1 11316 0.84 MB/sec warmup 93 sec latency 230.220 ms 2024-01-21T22:28:36.917 INFO:tasks.workunit.client.0.smithi138.stdout: 1 11440 0.84 MB/sec warmup 94 sec latency 206.441 ms 2024-01-21T22:28:37.917 INFO:tasks.workunit.client.0.smithi138.stdout: 1 11572 0.83 MB/sec warmup 95 sec latency 139.743 ms 2024-01-21T22:28:38.917 INFO:tasks.workunit.client.0.smithi138.stdout: 1 11721 0.84 MB/sec warmup 96 sec latency 119.108 ms 2024-01-21T22:28:39.917 INFO:tasks.workunit.client.0.smithi138.stdout: 1 11867 0.83 MB/sec warmup 97 sec latency 105.807 ms 2024-01-21T22:28:40.917 INFO:tasks.workunit.client.0.smithi138.stdout: 1 12049 0.85 MB/sec warmup 98 sec latency 191.113 ms 2024-01-21T22:28:41.917 INFO:tasks.workunit.client.0.smithi138.stdout: 1 12102 0.84 MB/sec warmup 99 sec latency 204.433 ms 2024-01-21T22:28:42.917 INFO:tasks.workunit.client.0.smithi138.stdout: 1 12243 0.84 MB/sec warmup 100 sec latency 204.599 ms 2024-01-21T22:28:43.918 INFO:tasks.workunit.client.0.smithi138.stdout: 1 12289 0.83 MB/sec warmup 101 sec latency 434.737 ms 2024-01-21T22:28:44.918 INFO:tasks.workunit.client.0.smithi138.stdout: 1 12425 0.82 MB/sec warmup 102 sec latency 238.389 ms 2024-01-21T22:28:45.918 INFO:tasks.workunit.client.0.smithi138.stdout: 1 12743 0.82 MB/sec warmup 103 sec latency 175.609 ms 2024-01-21T22:28:46.918 INFO:tasks.workunit.client.0.smithi138.stdout: 1 13021 0.82 MB/sec warmup 104 sec latency 91.235 ms 2024-01-21T22:28:47.918 INFO:tasks.workunit.client.0.smithi138.stdout: 1 13093 0.81 MB/sec warmup 105 sec latency 136.989 ms 2024-01-21T22:28:48.918 INFO:tasks.workunit.client.0.smithi138.stdout: 1 13223 0.81 MB/sec warmup 106 sec latency 141.229 ms 2024-01-21T22:28:49.918 INFO:tasks.workunit.client.0.smithi138.stdout: 1 13430 0.83 MB/sec warmup 107 sec latency 125.906 ms 2024-01-21T22:28:50.919 INFO:tasks.workunit.client.0.smithi138.stdout: 1 13619 0.84 MB/sec warmup 108 sec latency 161.714 ms 2024-01-21T22:28:51.919 INFO:tasks.workunit.client.0.smithi138.stdout: 1 13825 0.84 MB/sec warmup 109 sec latency 102.365 ms 2024-01-21T22:28:52.919 INFO:tasks.workunit.client.0.smithi138.stdout: 1 14001 0.84 MB/sec warmup 110 sec latency 129.767 ms 2024-01-21T22:28:53.919 INFO:tasks.workunit.client.0.smithi138.stdout: 1 14142 0.86 MB/sec warmup 111 sec latency 154.987 ms 2024-01-21T22:28:54.919 INFO:tasks.workunit.client.0.smithi138.stdout: 1 14161 0.85 MB/sec warmup 112 sec latency 934.739 ms 2024-01-21T22:28:55.919 INFO:tasks.workunit.client.0.smithi138.stdout: 1 14280 0.85 MB/sec warmup 113 sec latency 1058.262 ms 2024-01-21T22:28:56.919 INFO:tasks.workunit.client.0.smithi138.stdout: 1 14406 0.85 MB/sec warmup 114 sec latency 99.212 ms 2024-01-21T22:28:57.919 INFO:tasks.workunit.client.0.smithi138.stdout: 1 14440 0.84 MB/sec warmup 115 sec latency 121.679 ms 2024-01-21T22:28:58.920 INFO:tasks.workunit.client.0.smithi138.stdout: 1 14491 0.84 MB/sec warmup 116 sec latency 91.108 ms 2024-01-21T22:28:59.920 INFO:tasks.workunit.client.0.smithi138.stdout: 1 14566 0.83 MB/sec warmup 117 sec latency 88.531 ms 2024-01-21T22:29:00.920 INFO:tasks.workunit.client.0.smithi138.stdout: 1 14598 0.83 MB/sec warmup 118 sec latency 152.299 ms 2024-01-21T22:29:01.920 INFO:tasks.workunit.client.0.smithi138.stdout: 1 14645 0.82 MB/sec warmup 119 sec latency 106.459 ms 2024-01-21T22:29:03.920 INFO:tasks.workunit.client.0.smithi138.stdout: 1 14752 0.02 MB/sec execute 1 sec latency 176.383 ms 2024-01-21T22:29:04.920 INFO:tasks.workunit.client.0.smithi138.stdout: 1 14791 0.01 MB/sec execute 2 sec latency 145.912 ms 2024-01-21T22:29:05.685 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:29:05.693 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:29:05.921 INFO:tasks.workunit.client.0.smithi138.stdout: 1 14845 0.04 MB/sec execute 3 sec latency 339.565 ms 2024-01-21T22:29:06.921 INFO:tasks.workunit.client.0.smithi138.stdout: 1 14933 0.09 MB/sec execute 4 sec latency 454.238 ms 2024-01-21T22:29:07.921 INFO:tasks.workunit.client.0.smithi138.stdout: 1 15051 0.12 MB/sec execute 5 sec latency 149.822 ms 2024-01-21T22:29:08.921 INFO:tasks.workunit.client.0.smithi138.stdout: 1 15174 0.14 MB/sec execute 6 sec latency 111.417 ms 2024-01-21T22:29:09.921 INFO:tasks.workunit.client.0.smithi138.stdout: 1 15264 0.28 MB/sec execute 7 sec latency 163.364 ms 2024-01-21T22:29:10.921 INFO:tasks.workunit.client.0.smithi138.stdout: 1 15363 0.26 MB/sec execute 8 sec latency 115.818 ms 2024-01-21T22:29:11.921 INFO:tasks.workunit.client.0.smithi138.stdout: 1 15475 0.24 MB/sec execute 9 sec latency 99.228 ms 2024-01-21T22:29:12.922 INFO:tasks.workunit.client.0.smithi138.stdout: 1 15631 0.51 MB/sec execute 10 sec latency 104.664 ms 2024-01-21T22:29:13.922 INFO:tasks.workunit.client.0.smithi138.stdout: 1 15691 0.46 MB/sec execute 11 sec latency 203.400 ms 2024-01-21T22:29:14.922 INFO:tasks.workunit.client.0.smithi138.stdout: 1 15787 0.43 MB/sec execute 12 sec latency 101.184 ms 2024-01-21T22:29:15.922 INFO:tasks.workunit.client.0.smithi138.stdout: 1 15887 0.41 MB/sec execute 13 sec latency 103.855 ms 2024-01-21T22:29:16.922 INFO:tasks.workunit.client.0.smithi138.stdout: 1 16037 0.39 MB/sec execute 14 sec latency 141.225 ms 2024-01-21T22:29:17.922 INFO:tasks.workunit.client.0.smithi138.stdout: 1 16151 0.37 MB/sec execute 15 sec latency 91.295 ms 2024-01-21T22:29:18.922 INFO:tasks.workunit.client.0.smithi138.stdout: 1 16491 0.43 MB/sec execute 16 sec latency 223.266 ms 2024-01-21T22:29:19.922 INFO:tasks.workunit.client.0.smithi138.stdout: 1 16592 0.41 MB/sec execute 17 sec latency 127.058 ms 2024-01-21T22:29:20.923 INFO:tasks.workunit.client.0.smithi138.stdout: 1 16662 0.43 MB/sec execute 18 sec latency 156.017 ms 2024-01-21T22:29:21.923 INFO:tasks.workunit.client.0.smithi138.stdout: 1 16786 0.42 MB/sec execute 19 sec latency 121.373 ms 2024-01-21T22:29:22.923 INFO:tasks.workunit.client.0.smithi138.stdout: 1 16968 0.55 MB/sec execute 20 sec latency 281.168 ms 2024-01-21T22:29:23.923 INFO:tasks.workunit.client.0.smithi138.stdout: 1 17072 0.54 MB/sec execute 21 sec latency 612.140 ms 2024-01-21T22:29:24.923 INFO:tasks.workunit.client.0.smithi138.stdout: 1 17174 0.57 MB/sec execute 22 sec latency 157.524 ms 2024-01-21T22:29:25.923 INFO:tasks.workunit.client.0.smithi138.stdout: 1 17389 0.58 MB/sec execute 23 sec latency 112.067 ms 2024-01-21T22:29:26.923 INFO:tasks.workunit.client.0.smithi138.stdout: 1 17566 0.60 MB/sec execute 24 sec latency 114.117 ms 2024-01-21T22:29:27.924 INFO:tasks.workunit.client.0.smithi138.stdout: 1 17723 0.70 MB/sec execute 25 sec latency 166.864 ms 2024-01-21T22:29:28.924 INFO:tasks.workunit.client.0.smithi138.stdout: 1 17847 0.68 MB/sec execute 26 sec latency 149.146 ms 2024-01-21T22:29:29.924 INFO:tasks.workunit.client.0.smithi138.stdout: 1 17951 0.70 MB/sec execute 27 sec latency 113.855 ms 2024-01-21T22:29:30.924 INFO:tasks.workunit.client.0.smithi138.stdout: 1 17984 0.67 MB/sec execute 28 sec latency 156.399 ms 2024-01-21T22:29:31.924 INFO:tasks.workunit.client.0.smithi138.stdout: 1 18034 0.65 MB/sec execute 29 sec latency 156.892 ms 2024-01-21T22:29:32.924 INFO:tasks.workunit.client.0.smithi138.stdout: 1 18106 0.64 MB/sec execute 30 sec latency 87.379 ms 2024-01-21T22:29:33.924 INFO:tasks.workunit.client.0.smithi138.stdout: 1 18142 0.62 MB/sec execute 31 sec latency 111.424 ms 2024-01-21T22:29:34.924 INFO:tasks.workunit.client.0.smithi138.stdout: 1 18174 0.60 MB/sec execute 32 sec latency 253.932 ms 2024-01-21T22:29:35.823 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:29:35.831 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:29:35.924 INFO:tasks.workunit.client.0.smithi138.stdout: 1 18206 0.58 MB/sec execute 33 sec latency 271.312 ms 2024-01-21T22:29:36.925 INFO:tasks.workunit.client.0.smithi138.stdout: 1 18296 0.56 MB/sec execute 34 sec latency 94.531 ms 2024-01-21T22:29:37.925 INFO:tasks.workunit.client.0.smithi138.stdout: 1 18337 0.55 MB/sec execute 35 sec latency 168.651 ms 2024-01-21T22:29:38.925 INFO:tasks.workunit.client.0.smithi138.stdout: 1 18410 0.53 MB/sec execute 36 sec latency 208.225 ms 2024-01-21T22:29:39.925 INFO:tasks.workunit.client.0.smithi138.stdout: 1 18533 0.53 MB/sec execute 37 sec latency 165.336 ms 2024-01-21T22:29:40.925 INFO:tasks.workunit.client.0.smithi138.stdout: 1 18618 0.52 MB/sec execute 38 sec latency 137.167 ms 2024-01-21T22:29:41.925 INFO:tasks.workunit.client.0.smithi138.stdout: 1 18712 0.51 MB/sec execute 39 sec latency 209.435 ms 2024-01-21T22:29:42.925 INFO:tasks.workunit.client.0.smithi138.stdout: 1 18851 0.53 MB/sec execute 40 sec latency 120.908 ms 2024-01-21T22:29:43.925 INFO:tasks.workunit.client.0.smithi138.stdout: 1 18975 0.52 MB/sec execute 41 sec latency 97.005 ms 2024-01-21T22:29:44.926 INFO:tasks.workunit.client.0.smithi138.stdout: 1 19170 0.58 MB/sec execute 42 sec latency 150.932 ms 2024-01-21T22:29:45.926 INFO:tasks.workunit.client.0.smithi138.stdout: 1 19302 0.56 MB/sec execute 43 sec latency 179.460 ms 2024-01-21T22:29:46.926 INFO:tasks.workunit.client.0.smithi138.stdout: 1 19508 0.56 MB/sec execute 44 sec latency 121.001 ms 2024-01-21T22:29:47.926 INFO:tasks.workunit.client.0.smithi138.stdout: 1 19808 0.55 MB/sec execute 45 sec latency 139.664 ms 2024-01-21T22:29:48.926 INFO:tasks.workunit.client.0.smithi138.stdout: 1 20138 0.57 MB/sec execute 46 sec latency 90.509 ms 2024-01-21T22:29:49.926 INFO:tasks.workunit.client.0.smithi138.stdout: 1 20179 0.56 MB/sec execute 47 sec latency 604.389 ms 2024-01-21T22:29:50.926 INFO:tasks.workunit.client.0.smithi138.stdout: 1 20225 0.55 MB/sec execute 48 sec latency 198.241 ms 2024-01-21T22:29:51.926 INFO:tasks.workunit.client.0.smithi138.stdout: 1 20341 0.55 MB/sec execute 49 sec latency 209.123 ms 2024-01-21T22:29:52.927 INFO:tasks.workunit.client.0.smithi138.stdout: 1 20549 0.60 MB/sec execute 50 sec latency 135.911 ms 2024-01-21T22:29:53.927 INFO:tasks.workunit.client.0.smithi138.stdout: 1 20714 0.62 MB/sec execute 51 sec latency 94.554 ms 2024-01-21T22:29:54.927 INFO:tasks.workunit.client.0.smithi138.stdout: 1 20956 0.62 MB/sec execute 52 sec latency 123.076 ms 2024-01-21T22:29:55.927 INFO:tasks.workunit.client.0.smithi138.stdout: 1 21120 0.63 MB/sec execute 53 sec latency 174.375 ms 2024-01-21T22:29:56.927 INFO:tasks.workunit.client.0.smithi138.stdout: 1 21290 0.68 MB/sec execute 54 sec latency 197.673 ms 2024-01-21T22:29:57.927 INFO:tasks.workunit.client.0.smithi138.stdout: 1 21398 0.67 MB/sec execute 55 sec latency 186.263 ms 2024-01-21T22:29:58.927 INFO:tasks.workunit.client.0.smithi138.stdout: 1 21477 0.66 MB/sec execute 56 sec latency 134.352 ms 2024-01-21T22:29:59.928 INFO:tasks.workunit.client.0.smithi138.stdout: 1 21564 0.67 MB/sec execute 57 sec latency 111.751 ms 2024-01-21T22:30:00.928 INFO:tasks.workunit.client.0.smithi138.stdout: 1 21605 0.65 MB/sec execute 58 sec latency 97.356 ms 2024-01-21T22:30:01.928 INFO:tasks.workunit.client.0.smithi138.stdout: 1 21652 0.64 MB/sec execute 59 sec latency 179.383 ms 2024-01-21T22:30:02.928 INFO:tasks.workunit.client.0.smithi138.stdout: 1 21708 0.64 MB/sec execute 60 sec latency 188.243 ms 2024-01-21T22:30:03.928 INFO:tasks.workunit.client.0.smithi138.stdout: 1 21748 0.63 MB/sec execute 61 sec latency 165.973 ms 2024-01-21T22:30:04.928 INFO:tasks.workunit.client.0.smithi138.stdout: 1 21797 0.62 MB/sec execute 62 sec latency 93.401 ms 2024-01-21T22:30:05.928 INFO:tasks.workunit.client.0.smithi138.stdout: 1 21882 0.61 MB/sec execute 63 sec latency 94.082 ms 2024-01-21T22:30:05.953 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:30:05.961 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:30:06.928 INFO:tasks.workunit.client.0.smithi138.stdout: 1 21917 0.60 MB/sec execute 64 sec latency 218.477 ms 2024-01-21T22:30:07.929 INFO:tasks.workunit.client.0.smithi138.stdout: 1 21973 0.59 MB/sec execute 65 sec latency 357.849 ms 2024-01-21T22:30:08.929 INFO:tasks.workunit.client.0.smithi138.stdout: 1 22076 0.59 MB/sec execute 66 sec latency 175.186 ms 2024-01-21T22:30:09.929 INFO:tasks.workunit.client.0.smithi138.stdout: 1 22198 0.58 MB/sec execute 67 sec latency 146.784 ms 2024-01-21T22:30:10.929 INFO:tasks.workunit.client.0.smithi138.stdout: 1 22303 0.58 MB/sec execute 68 sec latency 190.566 ms 2024-01-21T22:30:11.929 INFO:tasks.workunit.client.0.smithi138.stdout: 1 22466 0.58 MB/sec execute 69 sec latency 219.424 ms 2024-01-21T22:30:12.929 INFO:tasks.workunit.client.0.smithi138.stdout: 1 22586 0.58 MB/sec execute 70 sec latency 89.156 ms 2024-01-21T22:30:13.929 INFO:tasks.workunit.client.0.smithi138.stdout: 1 22812 0.61 MB/sec execute 71 sec latency 124.783 ms 2024-01-21T22:30:14.930 INFO:tasks.workunit.client.0.smithi138.stdout: 1 22983 0.60 MB/sec execute 72 sec latency 123.236 ms 2024-01-21T22:30:15.931 INFO:tasks.workunit.client.0.smithi138.stdout: 1 23185 0.60 MB/sec execute 73 sec latency 154.599 ms 2024-01-21T22:30:16.930 INFO:tasks.workunit.client.0.smithi138.stdout: 1 23318 0.59 MB/sec execute 74 sec latency 450.619 ms 2024-01-21T22:30:17.930 INFO:tasks.workunit.client.0.smithi138.stdout: 1 23622 0.60 MB/sec execute 75 sec latency 526.443 ms 2024-01-21T22:30:18.930 INFO:tasks.workunit.client.0.smithi138.stdout: 1 23726 0.60 MB/sec execute 76 sec latency 143.763 ms 2024-01-21T22:30:19.930 INFO:tasks.workunit.client.0.smithi138.stdout: 1 23803 0.60 MB/sec execute 77 sec latency 115.362 ms 2024-01-21T22:30:20.930 INFO:tasks.workunit.client.0.smithi138.stdout: 1 23933 0.59 MB/sec execute 78 sec latency 137.274 ms 2024-01-21T22:30:21.931 INFO:tasks.workunit.client.0.smithi138.stdout: 1 24204 0.63 MB/sec execute 79 sec latency 255.015 ms 2024-01-21T22:30:22.931 INFO:tasks.workunit.client.0.smithi138.stdout: 1 24330 0.64 MB/sec execute 80 sec latency 95.691 ms 2024-01-21T22:30:23.931 INFO:tasks.workunit.client.0.smithi138.stdout: 1 24598 0.65 MB/sec execute 81 sec latency 115.740 ms 2024-01-21T22:30:24.931 INFO:tasks.workunit.client.0.smithi138.stdout: 1 24702 0.64 MB/sec execute 82 sec latency 190.443 ms 2024-01-21T22:30:25.932 INFO:tasks.workunit.client.0.smithi138.stdout: 1 24855 0.67 MB/sec execute 83 sec latency 169.775 ms 2024-01-21T22:30:26.931 INFO:tasks.workunit.client.0.smithi138.stdout: 1 24934 0.67 MB/sec execute 84 sec latency 207.907 ms 2024-01-21T22:30:27.931 INFO:tasks.workunit.client.0.smithi138.stdout: 1 24979 0.66 MB/sec execute 85 sec latency 256.125 ms 2024-01-21T22:30:28.931 INFO:tasks.workunit.client.0.smithi138.stdout: 1 24992 0.65 MB/sec execute 86 sec latency 842.923 ms 2024-01-21T22:30:29.932 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25086 0.66 MB/sec execute 87 sec latency 101.967 ms 2024-01-21T22:30:30.932 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25120 0.65 MB/sec execute 88 sec latency 141.579 ms 2024-01-21T22:30:31.932 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25167 0.64 MB/sec execute 89 sec latency 143.922 ms 2024-01-21T22:30:32.932 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25242 0.64 MB/sec execute 90 sec latency 75.993 ms 2024-01-21T22:30:33.933 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25282 0.63 MB/sec execute 91 sec latency 112.133 ms 2024-01-21T22:30:34.932 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25337 0.62 MB/sec execute 92 sec latency 75.892 ms 2024-01-21T22:30:35.932 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25411 0.62 MB/sec execute 93 sec latency 114.021 ms 2024-01-21T22:30:36.092 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:30:36.100 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:30:36.933 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25448 0.61 MB/sec execute 94 sec latency 186.321 ms 2024-01-21T22:30:37.933 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25480 0.60 MB/sec execute 95 sec latency 232.700 ms 2024-01-21T22:30:38.933 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25597 0.60 MB/sec execute 96 sec latency 151.391 ms 2024-01-21T22:30:39.933 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25701 0.60 MB/sec execute 97 sec latency 137.802 ms 2024-01-21T22:30:40.933 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25853 0.59 MB/sec execute 98 sec latency 122.133 ms 2024-01-21T22:30:41.933 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25933 0.60 MB/sec execute 99 sec latency 293.375 ms 2024-01-21T22:30:42.935 INFO:tasks.workunit.client.0.smithi138.stdout: 1 26061 0.59 MB/sec execute 100 sec latency 106.782 ms 2024-01-21T22:30:43.934 INFO:tasks.workunit.client.0.smithi138.stdout: 1 26158 0.59 MB/sec execute 101 sec latency 185.961 ms 2024-01-21T22:30:44.934 INFO:tasks.workunit.client.0.smithi138.stdout: 1 26371 0.61 MB/sec execute 102 sec latency 192.828 ms 2024-01-21T22:30:45.934 INFO:tasks.workunit.client.0.smithi138.stdout: 1 26543 0.61 MB/sec execute 103 sec latency 134.510 ms 2024-01-21T22:30:46.934 INFO:tasks.workunit.client.0.smithi138.stdout: 1 26654 0.60 MB/sec execute 104 sec latency 133.580 ms 2024-01-21T22:30:47.934 INFO:tasks.workunit.client.0.smithi138.stdout: 1 26749 0.60 MB/sec execute 105 sec latency 99.903 ms 2024-01-21T22:30:48.934 INFO:tasks.workunit.client.0.smithi138.stdout: 1 26935 0.60 MB/sec execute 106 sec latency 154.477 ms 2024-01-21T22:30:49.934 INFO:tasks.workunit.client.0.smithi138.stdout: 1 27171 0.60 MB/sec execute 107 sec latency 186.975 ms 2024-01-21T22:30:50.935 INFO:tasks.workunit.client.0.smithi138.stdout: 1 27229 0.60 MB/sec execute 108 sec latency 774.537 ms 2024-01-21T22:30:51.935 INFO:tasks.workunit.client.0.smithi138.stdout: 1 27283 0.59 MB/sec execute 109 sec latency 231.260 ms 2024-01-21T22:30:52.935 INFO:tasks.workunit.client.0.smithi138.stdout: 1 27366 0.59 MB/sec execute 110 sec latency 127.553 ms 2024-01-21T22:30:53.935 INFO:tasks.workunit.client.0.smithi138.stdout: 1 27482 0.59 MB/sec execute 111 sec latency 126.584 ms 2024-01-21T22:30:54.935 INFO:tasks.workunit.client.0.smithi138.stdout: 1 27774 0.62 MB/sec execute 112 sec latency 101.550 ms 2024-01-21T22:30:55.935 INFO:tasks.workunit.client.0.smithi138.stdout: 1 27886 0.62 MB/sec execute 113 sec latency 122.771 ms 2024-01-21T22:30:56.936 INFO:tasks.workunit.client.0.smithi138.stdout: 1 28221 0.63 MB/sec execute 114 sec latency 103.660 ms 2024-01-21T22:30:57.936 INFO:tasks.workunit.client.0.smithi138.stdout: 1 28381 0.65 MB/sec execute 115 sec latency 131.521 ms 2024-01-21T22:30:58.936 INFO:tasks.workunit.client.0.smithi138.stdout: 1 28479 0.65 MB/sec execute 116 sec latency 204.789 ms 2024-01-21T22:30:59.936 INFO:tasks.workunit.client.0.smithi138.stdout: 1 28555 0.64 MB/sec execute 117 sec latency 147.980 ms 2024-01-21T22:31:00.936 INFO:tasks.workunit.client.0.smithi138.stdout: 1 28644 0.65 MB/sec execute 118 sec latency 94.234 ms 2024-01-21T22:31:01.936 INFO:tasks.workunit.client.0.smithi138.stdout: 1 28651 0.64 MB/sec execute 119 sec latency 725.069 ms 2024-01-21T22:31:02.936 INFO:tasks.workunit.client.0.smithi138.stdout: 1 28691 0.64 MB/sec execute 120 sec latency 172.175 ms 2024-01-21T22:31:03.936 INFO:tasks.workunit.client.0.smithi138.stdout: 1 28739 0.63 MB/sec execute 121 sec latency 113.517 ms 2024-01-21T22:31:04.936 INFO:tasks.workunit.client.0.smithi138.stdout: 1 28803 0.63 MB/sec execute 122 sec latency 95.492 ms 2024-01-21T22:31:05.937 INFO:tasks.workunit.client.0.smithi138.stdout: 1 28849 0.62 MB/sec execute 123 sec latency 125.807 ms 2024-01-21T22:31:06.291 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:31:06.299 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:31:06.937 INFO:tasks.workunit.client.0.smithi138.stdout: 1 28896 0.62 MB/sec execute 124 sec latency 95.926 ms 2024-01-21T22:31:07.937 INFO:tasks.workunit.client.0.smithi138.stdout: 1 28977 0.61 MB/sec execute 125 sec latency 173.494 ms 2024-01-21T22:31:08.938 INFO:tasks.workunit.client.0.smithi138.stdout: 1 29012 0.61 MB/sec execute 126 sec latency 163.540 ms 2024-01-21T22:31:09.937 INFO:tasks.workunit.client.0.smithi138.stdout: 1 29065 0.60 MB/sec execute 127 sec latency 396.522 ms 2024-01-21T22:31:10.937 INFO:tasks.workunit.client.0.smithi138.stdout: 1 29171 0.60 MB/sec execute 128 sec latency 122.641 ms 2024-01-21T22:31:11.937 INFO:tasks.workunit.client.0.smithi138.stdout: 1 29243 0.60 MB/sec execute 129 sec latency 226.629 ms 2024-01-21T22:31:12.937 INFO:tasks.workunit.client.0.smithi138.stdout: 1 29360 0.60 MB/sec execute 130 sec latency 123.611 ms 2024-01-21T22:31:13.938 INFO:tasks.workunit.client.0.smithi138.stdout: 1 29456 0.60 MB/sec execute 131 sec latency 164.921 ms 2024-01-21T22:31:14.938 INFO:tasks.workunit.client.0.smithi138.stdout: 1 29600 0.60 MB/sec execute 132 sec latency 103.242 ms 2024-01-21T22:31:15.938 INFO:tasks.workunit.client.0.smithi138.stdout: 1 29817 0.61 MB/sec execute 133 sec latency 142.817 ms 2024-01-21T22:31:16.938 INFO:tasks.workunit.client.0.smithi138.stdout: 1 29940 0.61 MB/sec execute 134 sec latency 116.565 ms 2024-01-21T22:31:17.938 INFO:tasks.workunit.client.0.smithi138.stdout: 1 30112 0.61 MB/sec execute 135 sec latency 143.905 ms 2024-01-21T22:31:18.938 INFO:tasks.workunit.client.0.smithi138.stdout: 1 30397 0.60 MB/sec execute 136 sec latency 127.035 ms 2024-01-21T22:31:19.938 INFO:tasks.workunit.client.0.smithi138.stdout: 1 30716 0.61 MB/sec execute 137 sec latency 152.094 ms 2024-01-21T22:31:20.939 INFO:tasks.workunit.client.0.smithi138.stdout: 1 30753 0.60 MB/sec execute 138 sec latency 503.135 ms 2024-01-21T22:31:21.939 INFO:tasks.workunit.client.0.smithi138.stdout: 1 30815 0.60 MB/sec execute 139 sec latency 192.551 ms 2024-01-21T22:31:22.939 INFO:tasks.workunit.client.0.smithi138.stdout: 1 30903 0.60 MB/sec execute 140 sec latency 111.519 ms 2024-01-21T22:31:23.939 INFO:tasks.workunit.client.0.smithi138.stdout: 1 31018 0.60 MB/sec execute 141 sec latency 138.977 ms 2024-01-21T22:31:24.939 INFO:tasks.workunit.client.0.smithi138.stdout: 1 31316 0.62 MB/sec execute 142 sec latency 82.568 ms 2024-01-21T22:31:25.939 INFO:tasks.workunit.client.0.smithi138.stdout: 1 31427 0.62 MB/sec execute 143 sec latency 121.905 ms 2024-01-21T22:31:26.939 INFO:tasks.workunit.client.0.smithi138.stdout: 1 31690 0.63 MB/sec execute 144 sec latency 109.236 ms 2024-01-21T22:31:27.939 INFO:tasks.workunit.client.0.smithi138.stdout: 1 31805 0.63 MB/sec execute 145 sec latency 142.946 ms 2024-01-21T22:31:28.940 INFO:tasks.workunit.client.0.smithi138.stdout: 1 32025 0.65 MB/sec execute 146 sec latency 138.276 ms 2024-01-21T22:31:29.940 INFO:tasks.workunit.client.0.smithi138.stdout: 1 32102 0.64 MB/sec execute 147 sec latency 137.058 ms 2024-01-21T22:31:30.940 INFO:tasks.workunit.client.0.smithi138.stdout: 1 32191 0.64 MB/sec execute 148 sec latency 126.980 ms 2024-01-21T22:31:31.940 INFO:tasks.workunit.client.0.smithi138.stdout: 1 32203 0.64 MB/sec execute 149 sec latency 828.464 ms 2024-01-21T22:31:32.940 INFO:tasks.workunit.client.0.smithi138.stdout: 1 32233 0.64 MB/sec execute 150 sec latency 968.499 ms 2024-01-21T22:31:33.940 INFO:tasks.workunit.client.0.smithi138.stdout: 1 32281 0.63 MB/sec execute 151 sec latency 81.269 ms 2024-01-21T22:31:34.941 INFO:tasks.workunit.client.0.smithi138.stdout: 1 32348 0.63 MB/sec execute 152 sec latency 131.167 ms 2024-01-21T22:31:35.941 INFO:tasks.workunit.client.0.smithi138.stdout: 1 32389 0.63 MB/sec execute 153 sec latency 111.847 ms 2024-01-21T22:31:36.420 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:31:36.428 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:31:36.941 INFO:tasks.workunit.client.0.smithi138.stdout: 1 32428 0.62 MB/sec execute 154 sec latency 168.995 ms 2024-01-21T22:31:37.941 INFO:tasks.workunit.client.0.smithi138.stdout: 1 32511 0.62 MB/sec execute 155 sec latency 128.739 ms 2024-01-21T22:31:38.941 INFO:tasks.workunit.client.0.smithi138.stdout: 1 32557 0.61 MB/sec execute 156 sec latency 191.811 ms 2024-01-21T22:31:39.941 INFO:tasks.workunit.client.0.smithi138.stdout: 1 32613 0.61 MB/sec execute 157 sec latency 277.274 ms 2024-01-21T22:31:40.941 INFO:tasks.workunit.client.0.smithi138.stdout: 1 32716 0.61 MB/sec execute 158 sec latency 163.299 ms 2024-01-21T22:31:41.941 INFO:tasks.workunit.client.0.smithi138.stdout: 1 32791 0.60 MB/sec execute 159 sec latency 283.012 ms 2024-01-21T22:31:42.941 INFO:tasks.workunit.client.0.smithi138.stdout: 1 32901 0.60 MB/sec execute 160 sec latency 146.454 ms 2024-01-21T22:31:43.941 INFO:tasks.workunit.client.0.smithi138.stdout: 1 33032 0.61 MB/sec execute 161 sec latency 119.561 ms 2024-01-21T22:31:44.942 INFO:tasks.workunit.client.0.smithi138.stdout: 1 33185 0.60 MB/sec execute 162 sec latency 100.032 ms 2024-01-21T22:31:45.942 INFO:tasks.workunit.client.0.smithi138.stdout: 1 33367 0.62 MB/sec execute 163 sec latency 177.336 ms 2024-01-21T22:31:46.942 INFO:tasks.workunit.client.0.smithi138.stdout: 1 33496 0.61 MB/sec execute 164 sec latency 110.118 ms 2024-01-21T22:31:47.942 INFO:tasks.workunit.client.0.smithi138.stdout: 1 33659 0.61 MB/sec execute 165 sec latency 156.324 ms 2024-01-21T22:31:48.942 INFO:tasks.workunit.client.0.smithi138.stdout: 1 33943 0.61 MB/sec execute 166 sec latency 124.330 ms 2024-01-21T22:31:49.942 INFO:tasks.workunit.client.0.smithi138.stdout: 1 34261 0.61 MB/sec execute 167 sec latency 274.616 ms 2024-01-21T22:31:50.943 INFO:tasks.workunit.client.0.smithi138.stdout: 1 34354 0.61 MB/sec execute 168 sec latency 205.008 ms 2024-01-21T22:31:51.942 INFO:tasks.workunit.client.0.smithi138.stdout: 1 34400 0.61 MB/sec execute 169 sec latency 182.712 ms 2024-01-21T22:31:52.943 INFO:tasks.workunit.client.0.smithi138.stdout: 1 34523 0.61 MB/sec execute 170 sec latency 143.398 ms 2024-01-21T22:31:53.943 INFO:tasks.workunit.client.0.smithi138.stdout: 1 34722 0.62 MB/sec execute 171 sec latency 134.509 ms 2024-01-21T22:31:54.943 INFO:tasks.workunit.client.0.smithi138.stdout: 1 34893 0.63 MB/sec execute 172 sec latency 80.072 ms 2024-01-21T22:31:55.943 INFO:tasks.workunit.client.0.smithi138.stdout: 1 34993 0.63 MB/sec execute 173 sec latency 110.714 ms 2024-01-21T22:31:56.943 INFO:tasks.workunit.client.0.smithi138.stdout: 1 35312 0.63 MB/sec execute 174 sec latency 132.267 ms 2024-01-21T22:31:57.943 INFO:tasks.workunit.client.0.smithi138.stdout: 1 35469 0.65 MB/sec execute 175 sec latency 124.975 ms 2024-01-21T22:31:58.943 INFO:tasks.workunit.client.0.smithi138.stdout: 1 35598 0.64 MB/sec execute 176 sec latency 102.307 ms 2024-01-21T22:31:59.944 INFO:tasks.workunit.client.0.smithi138.stdout: 1 35649 0.64 MB/sec execute 177 sec latency 178.882 ms 2024-01-21T22:32:00.944 INFO:tasks.workunit.client.0.smithi138.stdout: 1 35718 0.64 MB/sec execute 178 sec latency 450.973 ms 2024-01-21T22:32:01.944 INFO:tasks.workunit.client.0.smithi138.stdout: 1 35743 0.64 MB/sec execute 179 sec latency 216.285 ms 2024-01-21T22:32:02.944 INFO:tasks.workunit.client.0.smithi138.stdout: 1 35791 0.64 MB/sec execute 180 sec latency 122.619 ms 2024-01-21T22:32:03.944 INFO:tasks.workunit.client.0.smithi138.stdout: 1 35831 0.63 MB/sec execute 181 sec latency 214.797 ms 2024-01-21T22:32:04.944 INFO:tasks.workunit.client.0.smithi138.stdout: 1 35890 0.63 MB/sec execute 182 sec latency 117.600 ms 2024-01-21T22:32:05.944 INFO:tasks.workunit.client.0.smithi138.stdout: 1 35934 0.63 MB/sec execute 183 sec latency 125.312 ms 2024-01-21T22:32:06.593 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:32:06.601 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:32:06.945 INFO:tasks.workunit.client.0.smithi138.stdout: 1 35985 0.62 MB/sec execute 184 sec latency 93.726 ms 2024-01-21T22:32:07.945 INFO:tasks.workunit.client.0.smithi138.stdout: 1 36064 0.62 MB/sec execute 185 sec latency 81.577 ms 2024-01-21T22:32:08.945 INFO:tasks.workunit.client.0.smithi138.stdout: 1 36095 0.62 MB/sec execute 186 sec latency 228.339 ms 2024-01-21T22:32:09.945 INFO:tasks.workunit.client.0.smithi138.stdout: 1 36156 0.61 MB/sec execute 187 sec latency 240.281 ms 2024-01-21T22:32:10.946 INFO:tasks.workunit.client.0.smithi138.stdout: 1 36229 0.61 MB/sec execute 188 sec latency 309.803 ms 2024-01-21T22:32:11.945 INFO:tasks.workunit.client.0.smithi138.stdout: 1 36308 0.61 MB/sec execute 189 sec latency 321.249 ms 2024-01-21T22:32:12.945 INFO:tasks.workunit.client.0.smithi138.stdout: 1 36445 0.61 MB/sec execute 190 sec latency 126.964 ms 2024-01-21T22:32:13.946 INFO:tasks.workunit.client.0.smithi138.stdout: 1 36576 0.61 MB/sec execute 191 sec latency 115.502 ms 2024-01-21T22:32:14.946 INFO:tasks.workunit.client.0.smithi138.stdout: 1 36696 0.61 MB/sec execute 192 sec latency 94.675 ms 2024-01-21T22:32:15.946 INFO:tasks.workunit.client.0.smithi138.stdout: 1 36893 0.62 MB/sec execute 193 sec latency 179.677 ms 2024-01-21T22:32:16.946 INFO:tasks.workunit.client.0.smithi138.stdout: 1 37038 0.62 MB/sec execute 194 sec latency 127.742 ms 2024-01-21T22:32:17.946 INFO:tasks.workunit.client.0.smithi138.stdout: 1 37212 0.62 MB/sec execute 195 sec latency 120.913 ms 2024-01-21T22:32:18.946 INFO:tasks.workunit.client.0.smithi138.stdout: 1 37487 0.61 MB/sec execute 196 sec latency 133.047 ms 2024-01-21T22:32:19.946 INFO:tasks.workunit.client.0.smithi138.stdout: 1 37843 0.62 MB/sec execute 197 sec latency 134.683 ms 2024-01-21T22:32:20.946 INFO:tasks.workunit.client.0.smithi138.stdout: 1 37943 0.62 MB/sec execute 198 sec latency 129.784 ms 2024-01-21T22:32:21.947 INFO:tasks.workunit.client.0.smithi138.stdout: 1 37962 0.61 MB/sec execute 199 sec latency 643.411 ms 2024-01-21T22:32:22.947 INFO:tasks.workunit.client.0.smithi138.stdout: 1 38015 0.61 MB/sec execute 200 sec latency 685.444 ms 2024-01-21T22:32:23.947 INFO:tasks.workunit.client.0.smithi138.stdout: 1 38225 0.63 MB/sec execute 201 sec latency 135.768 ms 2024-01-21T22:32:24.947 INFO:tasks.workunit.client.0.smithi138.stdout: 1 38426 0.63 MB/sec execute 202 sec latency 83.829 ms 2024-01-21T22:32:25.947 INFO:tasks.workunit.client.0.smithi138.stdout: 1 38519 0.63 MB/sec execute 203 sec latency 111.505 ms 2024-01-21T22:32:26.947 INFO:tasks.workunit.client.0.smithi138.stdout: 1 38805 0.63 MB/sec execute 204 sec latency 136.371 ms 2024-01-21T22:32:27.947 INFO:tasks.workunit.client.0.smithi138.stdout: 1 38996 0.64 MB/sec execute 205 sec latency 146.149 ms 2024-01-21T22:32:28.948 INFO:tasks.workunit.client.0.smithi138.stdout: 1 39130 0.64 MB/sec execute 206 sec latency 135.497 ms 2024-01-21T22:32:29.948 INFO:tasks.workunit.client.0.smithi138.stdout: 1 39210 0.64 MB/sec execute 207 sec latency 121.904 ms 2024-01-21T22:32:30.948 INFO:tasks.workunit.client.0.smithi138.stdout: 1 39290 0.64 MB/sec execute 208 sec latency 119.939 ms 2024-01-21T22:32:31.948 INFO:tasks.workunit.client.0.smithi138.stdout: 1 39319 0.64 MB/sec execute 209 sec latency 197.392 ms 2024-01-21T22:32:32.948 INFO:tasks.workunit.client.0.smithi138.stdout: 1 39347 0.64 MB/sec execute 210 sec latency 379.464 ms 2024-01-21T22:32:33.948 INFO:tasks.workunit.client.0.smithi138.stdout: 1 39379 0.63 MB/sec execute 211 sec latency 515.311 ms 2024-01-21T22:32:34.948 INFO:tasks.workunit.client.0.smithi138.stdout: 1 39443 0.63 MB/sec execute 212 sec latency 100.437 ms 2024-01-21T22:32:35.948 INFO:tasks.workunit.client.0.smithi138.stdout: 1 39490 0.63 MB/sec execute 213 sec latency 130.455 ms 2024-01-21T22:32:36.731 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:32:36.741 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:32:36.949 INFO:tasks.workunit.client.0.smithi138.stdout: 1 39528 0.63 MB/sec execute 214 sec latency 191.085 ms 2024-01-21T22:32:37.951 INFO:tasks.workunit.client.0.smithi138.stdout: 1 39610 0.62 MB/sec execute 215 sec latency 89.707 ms 2024-01-21T22:32:38.949 INFO:tasks.workunit.client.0.smithi138.stdout: 1 39649 0.62 MB/sec execute 216 sec latency 157.206 ms 2024-01-21T22:32:39.949 INFO:tasks.workunit.client.0.smithi138.stdout: 1 39723 0.62 MB/sec execute 217 sec latency 301.688 ms 2024-01-21T22:32:40.949 INFO:tasks.workunit.client.0.smithi138.stdout: 1 39846 0.62 MB/sec execute 218 sec latency 150.473 ms 2024-01-21T22:32:41.949 INFO:tasks.workunit.client.0.smithi138.stdout: 1 39963 0.61 MB/sec execute 219 sec latency 123.266 ms 2024-01-21T22:32:42.950 INFO:tasks.workunit.client.0.smithi138.stdout: 1 40074 0.61 MB/sec execute 220 sec latency 189.406 ms 2024-01-21T22:32:43.950 INFO:tasks.workunit.client.0.smithi138.stdout: 1 40197 0.62 MB/sec execute 221 sec latency 186.433 ms 2024-01-21T22:32:44.950 INFO:tasks.workunit.client.0.smithi138.stdout: 1 40317 0.61 MB/sec execute 222 sec latency 93.747 ms 2024-01-21T22:32:45.950 INFO:tasks.workunit.client.0.smithi138.stdout: 1 40492 0.62 MB/sec execute 223 sec latency 127.221 ms 2024-01-21T22:32:46.950 INFO:tasks.workunit.client.0.smithi138.stdout: 1 40579 0.62 MB/sec execute 224 sec latency 230.358 ms 2024-01-21T22:32:47.950 INFO:tasks.workunit.client.0.smithi138.stdout: 1 40705 0.62 MB/sec execute 225 sec latency 192.322 ms 2024-01-21T22:32:48.950 INFO:tasks.workunit.client.0.smithi138.stdout: 1 40918 0.62 MB/sec execute 226 sec latency 126.266 ms 2024-01-21T22:32:49.950 INFO:tasks.workunit.client.0.smithi138.stdout: 1 41120 0.61 MB/sec execute 227 sec latency 364.748 ms 2024-01-21T22:32:50.950 INFO:tasks.workunit.client.0.smithi138.stdout: 1 41393 0.62 MB/sec execute 228 sec latency 168.063 ms 2024-01-21T22:32:51.951 INFO:tasks.workunit.client.0.smithi138.stdout: 1 41465 0.62 MB/sec execute 229 sec latency 189.058 ms 2024-01-21T22:32:52.951 INFO:tasks.workunit.client.0.smithi138.stdout: 1 41549 0.62 MB/sec execute 230 sec latency 121.342 ms 2024-01-21T22:32:53.951 INFO:tasks.workunit.client.0.smithi138.stdout: 1 41665 0.61 MB/sec execute 231 sec latency 135.085 ms 2024-01-21T22:32:54.951 INFO:tasks.workunit.client.0.smithi138.stdout: 1 41957 0.63 MB/sec execute 232 sec latency 105.534 ms 2024-01-21T22:32:55.951 INFO:tasks.workunit.client.0.smithi138.stdout: 1 42069 0.63 MB/sec execute 233 sec latency 108.991 ms 2024-01-21T22:32:56.951 INFO:tasks.workunit.client.0.smithi138.stdout: 1 42396 0.63 MB/sec execute 234 sec latency 101.430 ms 2024-01-21T22:32:57.951 INFO:tasks.workunit.client.0.smithi138.stdout: 1 42552 0.64 MB/sec execute 235 sec latency 202.761 ms 2024-01-21T22:32:58.952 INFO:tasks.workunit.client.0.smithi138.stdout: 1 42662 0.64 MB/sec execute 236 sec latency 137.142 ms 2024-01-21T22:32:59.952 INFO:tasks.workunit.client.0.smithi138.stdout: 1 42732 0.64 MB/sec execute 237 sec latency 161.930 ms 2024-01-21T22:33:00.952 INFO:tasks.workunit.client.0.smithi138.stdout: 1 42764 0.64 MB/sec execute 238 sec latency 384.252 ms 2024-01-21T22:33:01.952 INFO:tasks.workunit.client.0.smithi138.stdout: 1 42822 0.64 MB/sec execute 239 sec latency 351.432 ms 2024-01-21T22:33:02.952 INFO:tasks.workunit.client.0.smithi138.stdout: 1 42858 0.64 MB/sec execute 240 sec latency 150.042 ms 2024-01-21T22:33:03.952 INFO:tasks.workunit.client.0.smithi138.stdout: 1 42910 0.63 MB/sec execute 241 sec latency 90.882 ms 2024-01-21T22:33:04.952 INFO:tasks.workunit.client.0.smithi138.stdout: 1 42974 0.63 MB/sec execute 242 sec latency 113.688 ms 2024-01-21T22:33:05.952 INFO:tasks.workunit.client.0.smithi138.stdout: 1 43008 0.63 MB/sec execute 243 sec latency 150.654 ms 2024-01-21T22:33:06.870 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:33:06.878 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:33:06.953 INFO:tasks.workunit.client.0.smithi138.stdout: 1 43056 0.63 MB/sec execute 244 sec latency 89.373 ms 2024-01-21T22:33:07.953 INFO:tasks.workunit.client.0.smithi138.stdout: 1 43134 0.62 MB/sec execute 245 sec latency 122.513 ms 2024-01-21T22:33:08.953 INFO:tasks.workunit.client.0.smithi138.stdout: 1 43177 0.62 MB/sec execute 246 sec latency 121.893 ms 2024-01-21T22:33:09.953 INFO:tasks.workunit.client.0.smithi138.stdout: 1 43203 0.62 MB/sec execute 247 sec latency 224.031 ms 2024-01-21T22:33:10.953 INFO:tasks.workunit.client.0.smithi138.stdout: 1 43308 0.62 MB/sec execute 248 sec latency 187.133 ms 2024-01-21T22:33:11.953 INFO:tasks.workunit.client.0.smithi138.stdout: 1 43382 0.62 MB/sec execute 249 sec latency 144.107 ms 2024-01-21T22:33:12.953 INFO:tasks.workunit.client.0.smithi138.stdout: 1 43500 0.61 MB/sec execute 250 sec latency 132.963 ms 2024-01-21T22:33:13.954 INFO:tasks.workunit.client.0.smithi138.stdout: 1 43576 0.61 MB/sec execute 251 sec latency 199.201 ms 2024-01-21T22:33:14.954 INFO:tasks.workunit.client.0.smithi138.stdout: 1 43739 0.62 MB/sec execute 252 sec latency 143.629 ms 2024-01-21T22:33:15.954 INFO:tasks.workunit.client.0.smithi138.stdout: 1 43829 0.61 MB/sec execute 253 sec latency 172.893 ms 2024-01-21T22:33:16.954 INFO:tasks.workunit.client.0.smithi138.stdout: 1 44007 0.62 MB/sec execute 254 sec latency 187.515 ms 2024-01-21T22:33:17.954 INFO:tasks.workunit.client.0.smithi138.stdout: 1 44127 0.62 MB/sec execute 255 sec latency 167.614 ms 2024-01-21T22:33:18.955 INFO:tasks.workunit.client.0.smithi138.stdout: 1 44292 0.62 MB/sec execute 256 sec latency 122.493 ms 2024-01-21T22:33:19.954 INFO:tasks.workunit.client.0.smithi138.stdout: 1 44575 0.62 MB/sec execute 257 sec latency 140.018 ms 2024-01-21T22:33:20.955 INFO:tasks.workunit.client.0.smithi138.stdout: 1 44932 0.62 MB/sec execute 258 sec latency 137.860 ms 2024-01-21T22:33:21.955 INFO:tasks.workunit.client.0.smithi138.stdout: 1 45000 0.62 MB/sec execute 259 sec latency 151.591 ms 2024-01-21T22:33:22.955 INFO:tasks.workunit.client.0.smithi138.stdout: 1 45095 0.62 MB/sec execute 260 sec latency 104.908 ms 2024-01-21T22:33:23.955 INFO:tasks.workunit.client.0.smithi138.stdout: 1 45205 0.62 MB/sec execute 261 sec latency 128.456 ms 2024-01-21T22:33:24.858 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:09:09.051 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:33:24.954 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:09:09.212 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:33:24.955 INFO:tasks.workunit.client.0.smithi138.stdout: 1 45517 0.63 MB/sec execute 262 sec latency 81.001 ms 2024-01-21T22:33:24.967 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:09:09.225 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:33:25.003 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:09:09.196 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:33:25.016 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:09:09.273 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:33:25.024 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:09:09.282 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:33:25.403 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:09:09.660 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:33:25.410 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:09:09.668 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:33:25.520 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:09:09.714 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:33:25.955 INFO:tasks.workunit.client.0.smithi138.stdout: 1 45607 0.63 MB/sec execute 263 sec latency 118.617 ms 2024-01-21T22:33:26.955 INFO:tasks.workunit.client.0.smithi138.stdout: 1 45807 0.63 MB/sec execute 264 sec latency 167.921 ms 2024-01-21T22:33:27.955 INFO:tasks.workunit.client.0.smithi138.stdout: 1 45948 0.63 MB/sec execute 265 sec latency 499.814 ms 2024-01-21T22:33:28.955 INFO:tasks.workunit.client.0.smithi138.stdout: 1 45986 0.63 MB/sec execute 266 sec latency 1005.414 ms 2024-01-21T22:33:29.955 INFO:tasks.workunit.client.0.smithi138.stdout: 1 46142 0.64 MB/sec execute 267 sec latency 154.576 ms 2024-01-21T22:33:30.956 INFO:tasks.workunit.client.0.smithi138.stdout: 1 46277 0.64 MB/sec execute 268 sec latency 122.401 ms 2024-01-21T22:33:31.956 INFO:tasks.workunit.client.0.smithi138.stdout: 1 46368 0.64 MB/sec execute 269 sec latency 141.311 ms 2024-01-21T22:33:32.956 INFO:tasks.workunit.client.0.smithi138.stdout: 1 46400 0.64 MB/sec execute 270 sec latency 177.836 ms 2024-01-21T22:33:33.956 INFO:tasks.workunit.client.0.smithi138.stdout: 1 46456 0.63 MB/sec execute 271 sec latency 79.920 ms 2024-01-21T22:33:34.956 INFO:tasks.workunit.client.0.smithi138.stdout: 1 46532 0.63 MB/sec execute 272 sec latency 97.108 ms 2024-01-21T22:33:35.956 INFO:tasks.workunit.client.0.smithi138.stdout: 1 46575 0.63 MB/sec execute 273 sec latency 129.570 ms 2024-01-21T22:33:36.956 INFO:tasks.workunit.client.0.smithi138.stdout: 1 46622 0.63 MB/sec execute 274 sec latency 93.735 ms 2024-01-21T22:33:37.053 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:33:37.061 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:33:37.957 INFO:tasks.workunit.client.0.smithi138.stdout: 1 46678 0.63 MB/sec execute 275 sec latency 267.083 ms 2024-01-21T22:33:38.957 INFO:tasks.workunit.client.0.smithi138.stdout: 1 46711 0.62 MB/sec execute 276 sec latency 200.658 ms 2024-01-21T22:33:39.957 INFO:tasks.workunit.client.0.smithi138.stdout: 1 46743 0.62 MB/sec execute 277 sec latency 227.261 ms 2024-01-21T22:33:40.957 INFO:tasks.workunit.client.0.smithi138.stdout: 1 46836 0.62 MB/sec execute 278 sec latency 350.120 ms 2024-01-21T22:33:41.957 INFO:tasks.workunit.client.0.smithi138.stdout: 1 46928 0.62 MB/sec execute 279 sec latency 222.988 ms 2024-01-21T22:33:42.957 INFO:tasks.workunit.client.0.smithi138.stdout: 1 47055 0.62 MB/sec execute 280 sec latency 230.577 ms 2024-01-21T22:33:43.958 INFO:tasks.workunit.client.0.smithi138.stdout: 1 47188 0.62 MB/sec execute 281 sec latency 117.724 ms 2024-01-21T22:33:44.958 INFO:tasks.workunit.client.0.smithi138.stdout: 1 47330 0.62 MB/sec execute 282 sec latency 89.544 ms 2024-01-21T22:33:45.958 INFO:tasks.workunit.client.0.smithi138.stdout: 1 47547 0.63 MB/sec execute 283 sec latency 101.068 ms 2024-01-21T22:33:46.958 INFO:tasks.workunit.client.0.smithi138.stdout: 1 47666 0.62 MB/sec execute 284 sec latency 163.783 ms 2024-01-21T22:33:47.958 INFO:tasks.workunit.client.0.smithi138.stdout: 1 47839 0.62 MB/sec execute 285 sec latency 93.526 ms 2024-01-21T22:33:48.958 INFO:tasks.workunit.client.0.smithi138.stdout: 1 48114 0.62 MB/sec execute 286 sec latency 123.279 ms 2024-01-21T22:33:49.958 INFO:tasks.workunit.client.0.smithi138.stdout: 1 48453 0.62 MB/sec execute 287 sec latency 140.258 ms 2024-01-21T22:33:50.958 INFO:tasks.workunit.client.0.smithi138.stdout: 1 48548 0.62 MB/sec execute 288 sec latency 152.006 ms 2024-01-21T22:33:51.959 INFO:tasks.workunit.client.0.smithi138.stdout: 1 48601 0.62 MB/sec execute 289 sec latency 125.099 ms 2024-01-21T22:33:52.959 INFO:tasks.workunit.client.0.smithi138.stdout: 1 48742 0.62 MB/sec execute 290 sec latency 128.766 ms 2024-01-21T22:33:53.959 INFO:tasks.workunit.client.0.smithi138.stdout: 1 49023 0.63 MB/sec execute 291 sec latency 128.483 ms 2024-01-21T22:33:54.959 INFO:tasks.workunit.client.0.smithi138.stdout: 1 49077 0.63 MB/sec execute 292 sec latency 402.813 ms 2024-01-21T22:33:55.959 INFO:tasks.workunit.client.0.smithi138.stdout: 1 49086 0.63 MB/sec execute 293 sec latency 1326.785 ms 2024-01-21T22:33:56.959 INFO:tasks.workunit.client.0.smithi138.stdout: 1 49166 0.63 MB/sec execute 294 sec latency 180.115 ms 2024-01-21T22:33:57.959 INFO:tasks.workunit.client.0.smithi138.stdout: 1 49483 0.63 MB/sec execute 295 sec latency 114.828 ms 2024-01-21T22:33:58.959 INFO:tasks.workunit.client.0.smithi138.stdout: 1 49653 0.64 MB/sec execute 296 sec latency 91.526 ms 2024-01-21T22:33:59.959 INFO:tasks.workunit.client.0.smithi138.stdout: 1 49761 0.64 MB/sec execute 297 sec latency 177.175 ms 2024-01-21T22:34:00.960 INFO:tasks.workunit.client.0.smithi138.stdout: 1 49834 0.64 MB/sec execute 298 sec latency 99.113 ms 2024-01-21T22:34:01.960 INFO:tasks.workunit.client.0.smithi138.stdout: 1 49911 0.64 MB/sec execute 299 sec latency 156.769 ms 2024-01-21T22:34:02.960 INFO:tasks.workunit.client.0.smithi138.stdout: 1 49951 0.64 MB/sec execute 300 sec latency 105.740 ms 2024-01-21T22:34:03.960 INFO:tasks.workunit.client.0.smithi138.stdout: 1 50002 0.63 MB/sec execute 301 sec latency 92.710 ms 2024-01-21T22:34:04.960 INFO:tasks.workunit.client.0.smithi138.stdout: 1 50070 0.63 MB/sec execute 302 sec latency 93.976 ms 2024-01-21T22:34:05.960 INFO:tasks.workunit.client.0.smithi138.stdout: 1 50113 0.63 MB/sec execute 303 sec latency 193.593 ms 2024-01-21T22:34:06.960 INFO:tasks.workunit.client.0.smithi138.stdout: 1 50152 0.63 MB/sec execute 304 sec latency 213.020 ms 2024-01-21T22:34:07.181 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:34:07.190 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:34:07.676 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:09:51.934 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:34:07.678 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:09:51.872 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:34:07.679 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:09:51.937 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:34:07.700 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:09:51.893 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:34:07.702 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:09:51.960 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:34:07.707 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:09:51.965 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:34:07.735 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:09:51.993 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:34:07.757 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:09:52.015 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:34:07.955 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:09:52.148 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:34:07.960 INFO:tasks.workunit.client.0.smithi138.stdout: 1 50229 0.63 MB/sec execute 305 sec latency 104.975 ms 2024-01-21T22:34:08.961 INFO:tasks.workunit.client.0.smithi138.stdout: 1 50268 0.63 MB/sec execute 306 sec latency 168.884 ms 2024-01-21T22:34:09.961 INFO:tasks.workunit.client.0.smithi138.stdout: 1 50294 0.62 MB/sec execute 307 sec latency 252.597 ms 2024-01-21T22:34:10.961 INFO:tasks.workunit.client.0.smithi138.stdout: 1 50410 0.62 MB/sec execute 308 sec latency 197.965 ms 2024-01-21T22:34:11.961 INFO:tasks.workunit.client.0.smithi138.stdout: 1 50489 0.62 MB/sec execute 309 sec latency 160.364 ms 2024-01-21T22:34:12.961 INFO:tasks.workunit.client.0.smithi138.stdout: 1 50625 0.62 MB/sec execute 310 sec latency 165.134 ms 2024-01-21T22:34:13.961 INFO:tasks.workunit.client.0.smithi138.stdout: 1 50757 0.62 MB/sec execute 311 sec latency 117.431 ms 2024-01-21T22:34:14.962 INFO:tasks.workunit.client.0.smithi138.stdout: 1 50911 0.62 MB/sec execute 312 sec latency 89.194 ms 2024-01-21T22:34:15.962 INFO:tasks.workunit.client.0.smithi138.stdout: 1 51115 0.63 MB/sec execute 313 sec latency 173.896 ms 2024-01-21T22:34:16.962 INFO:tasks.workunit.client.0.smithi138.stdout: 1 51220 0.63 MB/sec execute 314 sec latency 137.519 ms 2024-01-21T22:34:17.962 INFO:tasks.workunit.client.0.smithi138.stdout: 1 51385 0.62 MB/sec execute 315 sec latency 125.668 ms 2024-01-21T22:34:18.962 INFO:tasks.workunit.client.0.smithi138.stdout: 1 51670 0.62 MB/sec execute 316 sec latency 119.679 ms 2024-01-21T22:34:19.962 INFO:tasks.workunit.client.0.smithi138.stdout: 1 51997 0.62 MB/sec execute 317 sec latency 130.398 ms 2024-01-21T22:34:20.962 INFO:tasks.workunit.client.0.smithi138.stdout: 1 52108 0.62 MB/sec execute 318 sec latency 111.533 ms 2024-01-21T22:34:21.963 INFO:tasks.workunit.client.0.smithi138.stdout: 1 52181 0.62 MB/sec execute 319 sec latency 183.688 ms 2024-01-21T22:34:22.963 INFO:tasks.workunit.client.0.smithi138.stdout: 1 52191 0.62 MB/sec execute 320 sec latency 530.320 ms 2024-01-21T22:34:23.963 INFO:tasks.workunit.client.0.smithi138.stdout: 1 52286 0.62 MB/sec execute 321 sec latency 383.302 ms 2024-01-21T22:34:24.963 INFO:tasks.workunit.client.0.smithi138.stdout: 1 52525 0.63 MB/sec execute 322 sec latency 296.547 ms 2024-01-21T22:34:25.963 INFO:tasks.workunit.client.0.smithi138.stdout: 1 52671 0.63 MB/sec execute 323 sec latency 99.785 ms 2024-01-21T22:34:26.963 INFO:tasks.workunit.client.0.smithi138.stdout: 1 52864 0.63 MB/sec execute 324 sec latency 134.032 ms 2024-01-21T22:34:27.963 INFO:tasks.workunit.client.0.smithi138.stdout: 1 53039 0.63 MB/sec execute 325 sec latency 146.429 ms 2024-01-21T22:34:28.964 INFO:tasks.workunit.client.0.smithi138.stdout: 1 53209 0.64 MB/sec execute 326 sec latency 143.016 ms 2024-01-21T22:34:29.964 INFO:tasks.workunit.client.0.smithi138.stdout: 1 53307 0.64 MB/sec execute 327 sec latency 108.162 ms 2024-01-21T22:34:30.964 INFO:tasks.workunit.client.0.smithi138.stdout: 1 53353 0.64 MB/sec execute 328 sec latency 189.809 ms 2024-01-21T22:34:31.964 INFO:tasks.workunit.client.0.smithi138.stdout: 1 53446 0.64 MB/sec execute 329 sec latency 133.661 ms 2024-01-21T22:34:32.964 INFO:tasks.workunit.client.0.smithi138.stdout: 1 53478 0.64 MB/sec execute 330 sec latency 263.918 ms 2024-01-21T22:34:33.965 INFO:tasks.workunit.client.0.smithi138.stdout: 1 53514 0.63 MB/sec execute 331 sec latency 165.809 ms 2024-01-21T22:34:34.964 INFO:tasks.workunit.client.0.smithi138.stdout: 1 53579 0.63 MB/sec execute 332 sec latency 81.246 ms 2024-01-21T22:34:35.965 INFO:tasks.workunit.client.0.smithi138.stdout: 1 53630 0.63 MB/sec execute 333 sec latency 108.266 ms 2024-01-21T22:34:36.965 INFO:tasks.workunit.client.0.smithi138.stdout: 1 53683 0.63 MB/sec execute 334 sec latency 108.611 ms 2024-01-21T22:34:37.321 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:34:37.329 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:34:37.965 INFO:tasks.workunit.client.0.smithi138.stdout: 1 53763 0.63 MB/sec execute 335 sec latency 111.365 ms 2024-01-21T22:34:38.965 INFO:tasks.workunit.client.0.smithi138.stdout: 1 53792 0.63 MB/sec execute 336 sec latency 291.566 ms 2024-01-21T22:34:39.965 INFO:tasks.workunit.client.0.smithi138.stdout: 1 53829 0.62 MB/sec execute 337 sec latency 114.558 ms 2024-01-21T22:34:40.965 INFO:tasks.workunit.client.0.smithi138.stdout: 1 53848 0.62 MB/sec execute 338 sec latency 327.783 ms 2024-01-21T22:34:41.965 INFO:tasks.workunit.client.0.smithi138.stdout: 1 53969 0.62 MB/sec execute 339 sec latency 236.504 ms 2024-01-21T22:34:42.966 INFO:tasks.workunit.client.0.smithi138.stdout: 1 54092 0.62 MB/sec execute 340 sec latency 160.218 ms 2024-01-21T22:34:43.966 INFO:tasks.workunit.client.0.smithi138.stdout: 1 54205 0.62 MB/sec execute 341 sec latency 107.384 ms 2024-01-21T22:34:44.967 INFO:tasks.workunit.client.0.smithi138.stdout: 1 54344 0.62 MB/sec execute 342 sec latency 112.017 ms 2024-01-21T22:34:45.966 INFO:tasks.workunit.client.0.smithi138.stdout: 1 54459 0.62 MB/sec execute 343 sec latency 95.694 ms 2024-01-21T22:34:46.966 INFO:tasks.workunit.client.0.smithi138.stdout: 1 54661 0.63 MB/sec execute 344 sec latency 177.997 ms 2024-01-21T22:34:47.966 INFO:tasks.workunit.client.0.smithi138.stdout: 1 54795 0.62 MB/sec execute 345 sec latency 116.762 ms 2024-01-21T22:34:48.966 INFO:tasks.workunit.client.0.smithi138.stdout: 1 55011 0.62 MB/sec execute 346 sec latency 119.304 ms 2024-01-21T22:34:49.967 INFO:tasks.workunit.client.0.smithi138.stdout: 1 55097 0.62 MB/sec execute 347 sec latency 667.286 ms 2024-01-21T22:34:50.967 INFO:tasks.workunit.client.0.smithi138.stdout: 1 55299 0.62 MB/sec execute 348 sec latency 871.199 ms 2024-01-21T22:34:51.967 INFO:tasks.workunit.client.0.smithi138.stdout: 1 55536 0.62 MB/sec execute 349 sec latency 151.296 ms 2024-01-21T22:34:52.967 INFO:tasks.workunit.client.0.smithi138.stdout: 1 55645 0.62 MB/sec execute 350 sec latency 107.599 ms 2024-01-21T22:34:53.967 INFO:tasks.workunit.client.0.smithi138.stdout: 1 55734 0.62 MB/sec execute 351 sec latency 140.199 ms 2024-01-21T22:34:54.967 INFO:tasks.workunit.client.0.smithi138.stdout: 1 55988 0.63 MB/sec execute 352 sec latency 115.838 ms 2024-01-21T22:34:55.967 INFO:tasks.workunit.client.0.smithi138.stdout: 1 56161 0.63 MB/sec execute 353 sec latency 95.710 ms 2024-01-21T22:34:56.967 INFO:tasks.workunit.client.0.smithi138.stdout: 1 56259 0.63 MB/sec execute 354 sec latency 99.820 ms 2024-01-21T22:34:57.967 INFO:tasks.workunit.client.0.smithi138.stdout: 1 56505 0.63 MB/sec execute 355 sec latency 190.265 ms 2024-01-21T22:34:58.968 INFO:tasks.workunit.client.0.smithi138.stdout: 1 56727 0.64 MB/sec execute 356 sec latency 86.134 ms 2024-01-21T22:34:59.968 INFO:tasks.workunit.client.0.smithi138.stdout: 1 56789 0.64 MB/sec execute 357 sec latency 131.789 ms 2024-01-21T22:35:00.968 INFO:tasks.workunit.client.0.smithi138.stdout: 1 56865 0.64 MB/sec execute 358 sec latency 142.093 ms 2024-01-21T22:35:01.968 INFO:tasks.workunit.client.0.smithi138.stdout: 1 56921 0.64 MB/sec execute 359 sec latency 103.585 ms 2024-01-21T22:35:02.968 INFO:tasks.workunit.client.0.smithi138.stdout: 1 57004 0.64 MB/sec execute 360 sec latency 155.116 ms 2024-01-21T22:35:03.968 INFO:tasks.workunit.client.0.smithi138.stdout: 1 57036 0.63 MB/sec execute 361 sec latency 224.457 ms 2024-01-21T22:35:04.969 INFO:tasks.workunit.client.0.smithi138.stdout: 1 57069 0.63 MB/sec execute 362 sec latency 267.421 ms 2024-01-21T22:35:05.969 INFO:tasks.workunit.client.0.smithi138.stdout: 1 57083 0.63 MB/sec execute 363 sec latency 527.038 ms 2024-01-21T22:35:06.969 INFO:tasks.workunit.client.0.smithi138.stdout: 1 57150 0.63 MB/sec execute 364 sec latency 144.834 ms 2024-01-21T22:35:07.497 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:35:07.505 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:35:07.969 INFO:tasks.workunit.client.0.smithi138.stdout: 1 57187 0.63 MB/sec execute 365 sec latency 105.794 ms 2024-01-21T22:35:08.969 INFO:tasks.workunit.client.0.smithi138.stdout: 1 57238 0.63 MB/sec execute 366 sec latency 88.101 ms 2024-01-21T22:35:09.969 INFO:tasks.workunit.client.0.smithi138.stdout: 1 57318 0.63 MB/sec execute 367 sec latency 95.147 ms 2024-01-21T22:35:10.972 INFO:tasks.workunit.client.0.smithi138.stdout: 1 57350 0.62 MB/sec execute 368 sec latency 166.825 ms 2024-01-21T22:35:11.969 INFO:tasks.workunit.client.0.smithi138.stdout: 1 57380 0.62 MB/sec execute 369 sec latency 375.655 ms 2024-01-21T22:35:12.970 INFO:tasks.workunit.client.0.smithi138.stdout: 1 57491 0.62 MB/sec execute 370 sec latency 139.252 ms 2024-01-21T22:35:13.970 INFO:tasks.workunit.client.0.smithi138.stdout: 1 57603 0.62 MB/sec execute 371 sec latency 192.847 ms 2024-01-21T22:35:14.970 INFO:tasks.workunit.client.0.smithi138.stdout: 1 57724 0.62 MB/sec execute 372 sec latency 127.834 ms 2024-01-21T22:35:15.970 INFO:tasks.workunit.client.0.smithi138.stdout: 1 57848 0.62 MB/sec execute 373 sec latency 163.720 ms 2024-01-21T22:35:16.970 INFO:tasks.workunit.client.0.smithi138.stdout: 1 57968 0.62 MB/sec execute 374 sec latency 89.377 ms 2024-01-21T22:35:17.970 INFO:tasks.workunit.client.0.smithi138.stdout: 1 58042 0.62 MB/sec execute 375 sec latency 156.014 ms 2024-01-21T22:35:18.970 INFO:tasks.workunit.client.0.smithi138.stdout: 1 58273 0.62 MB/sec execute 376 sec latency 173.510 ms 2024-01-21T22:35:19.971 INFO:tasks.workunit.client.0.smithi138.stdout: 1 58439 0.62 MB/sec execute 377 sec latency 103.622 ms 2024-01-21T22:35:20.971 INFO:tasks.workunit.client.0.smithi138.stdout: 1 58674 0.62 MB/sec execute 378 sec latency 125.918 ms 2024-01-21T22:35:21.971 INFO:tasks.workunit.client.0.smithi138.stdout: 1 59073 0.62 MB/sec execute 379 sec latency 150.282 ms 2024-01-21T22:35:22.971 INFO:tasks.workunit.client.0.smithi138.stdout: 1 59089 0.62 MB/sec execute 380 sec latency 789.036 ms 2024-01-21T22:35:23.971 INFO:tasks.workunit.client.0.smithi138.stdout: 1 59182 0.62 MB/sec execute 381 sec latency 207.648 ms 2024-01-21T22:35:24.971 INFO:tasks.workunit.client.0.smithi138.stdout: 1 59270 0.62 MB/sec execute 382 sec latency 118.009 ms 2024-01-21T22:35:25.971 INFO:tasks.workunit.client.0.smithi138.stdout: 1 59387 0.62 MB/sec execute 383 sec latency 140.815 ms 2024-01-21T22:35:26.971 INFO:tasks.workunit.client.0.smithi138.stdout: 1 59693 0.63 MB/sec execute 384 sec latency 88.891 ms 2024-01-21T22:35:27.972 INFO:tasks.workunit.client.0.smithi138.stdout: 1 59788 0.63 MB/sec execute 385 sec latency 116.024 ms 2024-01-21T22:35:28.972 INFO:tasks.workunit.client.0.smithi138.stdout: 1 60117 0.63 MB/sec execute 386 sec latency 124.613 ms 2024-01-21T22:35:29.972 INFO:tasks.workunit.client.0.smithi138.stdout: 1 60281 0.64 MB/sec execute 387 sec latency 117.888 ms 2024-01-21T22:35:30.972 INFO:tasks.workunit.client.0.smithi138.stdout: 1 60402 0.64 MB/sec execute 388 sec latency 130.191 ms 2024-01-21T22:35:31.972 INFO:tasks.workunit.client.0.smithi138.stdout: 1 60527 0.64 MB/sec execute 389 sec latency 86.939 ms 2024-01-21T22:35:32.972 INFO:tasks.workunit.client.0.smithi138.stdout: 1 60562 0.64 MB/sec execute 390 sec latency 229.792 ms 2024-01-21T22:35:33.972 INFO:tasks.workunit.client.0.smithi138.stdout: 1 60575 0.63 MB/sec execute 391 sec latency 661.452 ms 2024-01-21T22:35:34.973 INFO:tasks.workunit.client.0.smithi138.stdout: 1 60610 0.63 MB/sec execute 392 sec latency 330.022 ms 2024-01-21T22:35:35.973 INFO:tasks.workunit.client.0.smithi138.stdout: 1 60652 0.63 MB/sec execute 393 sec latency 136.796 ms 2024-01-21T22:35:36.973 INFO:tasks.workunit.client.0.smithi138.stdout: 1 60716 0.63 MB/sec execute 394 sec latency 100.846 ms 2024-01-21T22:35:37.637 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:35:37.645 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:35:37.973 INFO:tasks.workunit.client.0.smithi138.stdout: 1 60766 0.63 MB/sec execute 395 sec latency 100.507 ms 2024-01-21T22:35:38.973 INFO:tasks.workunit.client.0.smithi138.stdout: 1 60833 0.63 MB/sec execute 396 sec latency 102.696 ms 2024-01-21T22:35:39.973 INFO:tasks.workunit.client.0.smithi138.stdout: 1 60885 0.63 MB/sec execute 397 sec latency 218.603 ms 2024-01-21T22:35:40.973 INFO:tasks.workunit.client.0.smithi138.stdout: 1 60922 0.62 MB/sec execute 398 sec latency 237.724 ms 2024-01-21T22:35:41.974 INFO:tasks.workunit.client.0.smithi138.stdout: 1 60995 0.62 MB/sec execute 399 sec latency 319.611 ms 2024-01-21T22:35:42.974 INFO:tasks.workunit.client.0.smithi138.stdout: 1 61109 0.62 MB/sec execute 400 sec latency 243.239 ms 2024-01-21T22:35:43.974 INFO:tasks.workunit.client.0.smithi138.stdout: 1 61196 0.62 MB/sec execute 401 sec latency 183.456 ms 2024-01-21T22:35:44.974 INFO:tasks.workunit.client.0.smithi138.stdout: 1 61272 0.62 MB/sec execute 402 sec latency 141.949 ms 2024-01-21T22:35:45.974 INFO:tasks.workunit.client.0.smithi138.stdout: 1 61397 0.62 MB/sec execute 403 sec latency 108.562 ms 2024-01-21T22:35:46.974 INFO:tasks.workunit.client.0.smithi138.stdout: 1 61549 0.62 MB/sec execute 404 sec latency 92.408 ms 2024-01-21T22:35:47.974 INFO:tasks.workunit.client.0.smithi138.stdout: 1 61781 0.63 MB/sec execute 405 sec latency 116.971 ms 2024-01-21T22:35:48.975 INFO:tasks.workunit.client.0.smithi138.stdout: 1 61969 0.63 MB/sec execute 406 sec latency 110.184 ms 2024-01-21T22:35:49.975 INFO:tasks.workunit.client.0.smithi138.stdout: 1 62140 0.62 MB/sec execute 407 sec latency 144.420 ms 2024-01-21T22:35:50.975 INFO:tasks.workunit.client.0.smithi138.stdout: 1 62514 0.62 MB/sec execute 408 sec latency 136.236 ms 2024-01-21T22:35:51.975 INFO:tasks.workunit.client.0.smithi138.stdout: 1 62698 0.62 MB/sec execute 409 sec latency 244.398 ms 2024-01-21T22:35:52.975 INFO:tasks.workunit.client.0.smithi138.stdout: 1 62724 0.62 MB/sec execute 410 sec latency 397.982 ms 2024-01-21T22:35:53.975 INFO:tasks.workunit.client.0.smithi138.stdout: 1 62785 0.62 MB/sec execute 411 sec latency 111.766 ms 2024-01-21T22:35:54.975 INFO:tasks.workunit.client.0.smithi138.stdout: 1 62925 0.62 MB/sec execute 412 sec latency 120.684 ms 2024-01-21T22:35:55.976 INFO:tasks.workunit.client.0.smithi138.stdout: 1 63221 0.63 MB/sec execute 413 sec latency 113.340 ms 2024-01-21T22:35:56.976 INFO:tasks.workunit.client.0.smithi138.stdout: 1 63326 0.63 MB/sec execute 414 sec latency 96.126 ms 2024-01-21T22:35:57.976 INFO:tasks.workunit.client.0.smithi138.stdout: 1 63502 0.63 MB/sec execute 415 sec latency 170.460 ms 2024-01-21T22:35:58.976 INFO:tasks.workunit.client.0.smithi138.stdout: 1 63690 0.63 MB/sec execute 416 sec latency 131.107 ms 2024-01-21T22:35:59.976 INFO:tasks.workunit.client.0.smithi138.stdout: 1 63854 0.64 MB/sec execute 417 sec latency 93.827 ms 2024-01-21T22:36:00.976 INFO:tasks.workunit.client.0.smithi138.stdout: 1 63943 0.64 MB/sec execute 418 sec latency 99.384 ms 2024-01-21T22:36:01.976 INFO:tasks.workunit.client.0.smithi138.stdout: 1 63993 0.64 MB/sec execute 419 sec latency 231.825 ms 2024-01-21T22:36:02.976 INFO:tasks.workunit.client.0.smithi138.stdout: 1 64085 0.64 MB/sec execute 420 sec latency 168.854 ms 2024-01-21T22:36:03.976 INFO:tasks.workunit.client.0.smithi138.stdout: 1 64098 0.64 MB/sec execute 421 sec latency 564.991 ms 2024-01-21T22:36:04.977 INFO:tasks.workunit.client.0.smithi138.stdout: 1 64135 0.63 MB/sec execute 422 sec latency 121.639 ms 2024-01-21T22:36:05.977 INFO:tasks.workunit.client.0.smithi138.stdout: 1 64188 0.63 MB/sec execute 423 sec latency 103.284 ms 2024-01-21T22:36:06.977 INFO:tasks.workunit.client.0.smithi138.stdout: 1 64246 0.63 MB/sec execute 424 sec latency 125.985 ms 2024-01-21T22:36:07.779 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:36:07.788 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:36:07.977 INFO:tasks.workunit.client.0.smithi138.stdout: 1 64288 0.63 MB/sec execute 425 sec latency 185.415 ms 2024-01-21T22:36:08.977 INFO:tasks.workunit.client.0.smithi138.stdout: 1 64337 0.63 MB/sec execute 426 sec latency 80.735 ms 2024-01-21T22:36:09.977 INFO:tasks.workunit.client.0.smithi138.stdout: 1 64419 0.63 MB/sec execute 427 sec latency 130.125 ms 2024-01-21T22:36:10.977 INFO:tasks.workunit.client.0.smithi138.stdout: 1 64458 0.63 MB/sec execute 428 sec latency 167.862 ms 2024-01-21T22:36:11.978 INFO:tasks.workunit.client.0.smithi138.stdout: 1 64479 0.62 MB/sec execute 429 sec latency 359.496 ms 2024-01-21T22:36:12.978 INFO:tasks.workunit.client.0.smithi138.stdout: 1 64591 0.62 MB/sec execute 430 sec latency 190.997 ms 2024-01-21T22:36:13.978 INFO:tasks.workunit.client.0.smithi138.stdout: 1 64696 0.62 MB/sec execute 431 sec latency 167.409 ms 2024-01-21T22:36:14.978 INFO:tasks.workunit.client.0.smithi138.stdout: 1 64843 0.62 MB/sec execute 432 sec latency 124.656 ms 2024-01-21T22:36:15.978 INFO:tasks.workunit.client.0.smithi138.stdout: 1 64938 0.62 MB/sec execute 433 sec latency 170.622 ms 2024-01-21T22:36:16.978 INFO:tasks.workunit.client.0.smithi138.stdout: 1 65076 0.62 MB/sec execute 434 sec latency 108.774 ms 2024-01-21T22:36:17.978 INFO:tasks.workunit.client.0.smithi138.stdout: 1 65149 0.62 MB/sec execute 435 sec latency 221.287 ms 2024-01-21T22:36:18.979 INFO:tasks.workunit.client.0.smithi138.stdout: 1 65375 0.63 MB/sec execute 436 sec latency 123.653 ms 2024-01-21T22:36:19.979 INFO:tasks.workunit.client.0.smithi138.stdout: 1 65556 0.62 MB/sec execute 437 sec latency 107.254 ms 2024-01-21T22:36:20.979 INFO:tasks.workunit.client.0.smithi138.stdout: 1 65826 0.62 MB/sec execute 438 sec latency 116.261 ms 2024-01-21T22:36:21.979 INFO:tasks.workunit.client.0.smithi138.stdout: 1 66177 0.63 MB/sec execute 439 sec latency 140.098 ms 2024-01-21T22:36:22.979 INFO:tasks.workunit.client.0.smithi138.stdout: 1 66272 0.62 MB/sec execute 440 sec latency 159.994 ms 2024-01-21T22:36:23.979 INFO:tasks.workunit.client.0.smithi138.stdout: 1 66349 0.62 MB/sec execute 441 sec latency 131.777 ms 2024-01-21T22:36:24.979 INFO:tasks.workunit.client.0.smithi138.stdout: 1 66477 0.62 MB/sec execute 442 sec latency 148.544 ms 2024-01-21T22:36:25.980 INFO:tasks.workunit.client.0.smithi138.stdout: 1 66764 0.63 MB/sec execute 443 sec latency 125.537 ms 2024-01-21T22:36:26.979 INFO:tasks.workunit.client.0.smithi138.stdout: 1 66880 0.63 MB/sec execute 444 sec latency 99.841 ms 2024-01-21T22:36:27.980 INFO:tasks.workunit.client.0.smithi138.stdout: 1 67092 0.63 MB/sec execute 445 sec latency 119.790 ms 2024-01-21T22:36:28.980 INFO:tasks.workunit.client.0.smithi138.stdout: 1 67258 0.63 MB/sec execute 446 sec latency 124.424 ms 2024-01-21T22:36:29.980 INFO:tasks.workunit.client.0.smithi138.stdout: 1 67489 0.64 MB/sec execute 447 sec latency 132.770 ms 2024-01-21T22:36:30.980 INFO:tasks.workunit.client.0.smithi138.stdout: 1 67581 0.64 MB/sec execute 448 sec latency 86.046 ms 2024-01-21T22:36:31.980 INFO:tasks.workunit.client.0.smithi138.stdout: 1 67653 0.64 MB/sec execute 449 sec latency 165.380 ms 2024-01-21T22:36:32.980 INFO:tasks.workunit.client.0.smithi138.stdout: 1 67662 0.64 MB/sec execute 450 sec latency 799.839 ms 2024-01-21T22:36:33.981 INFO:tasks.workunit.client.0.smithi138.stdout: 1 67666 0.64 MB/sec execute 451 sec latency 1567.856 ms 2024-01-21T22:36:34.981 INFO:tasks.workunit.client.0.smithi138.stdout: 1 67712 0.63 MB/sec execute 452 sec latency 258.313 ms 2024-01-21T22:36:35.981 INFO:tasks.workunit.client.0.smithi138.stdout: 1 67781 0.63 MB/sec execute 453 sec latency 78.694 ms 2024-01-21T22:36:36.981 INFO:tasks.workunit.client.0.smithi138.stdout: 1 67821 0.63 MB/sec execute 454 sec latency 113.249 ms 2024-01-21T22:36:37.945 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:36:37.954 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:36:37.981 INFO:tasks.workunit.client.0.smithi138.stdout: 1 67874 0.63 MB/sec execute 455 sec latency 93.500 ms 2024-01-21T22:36:38.981 INFO:tasks.workunit.client.0.smithi138.stdout: 1 67957 0.63 MB/sec execute 456 sec latency 101.780 ms 2024-01-21T22:36:39.981 INFO:tasks.workunit.client.0.smithi138.stdout: 1 67990 0.63 MB/sec execute 457 sec latency 208.106 ms 2024-01-21T22:36:40.981 INFO:tasks.workunit.client.0.smithi138.stdout: 1 68023 0.63 MB/sec execute 458 sec latency 387.349 ms 2024-01-21T22:36:41.982 INFO:tasks.workunit.client.0.smithi138.stdout: 1 68144 0.63 MB/sec execute 459 sec latency 152.574 ms 2024-01-21T22:36:42.982 INFO:tasks.workunit.client.0.smithi138.stdout: 1 68243 0.62 MB/sec execute 460 sec latency 307.030 ms 2024-01-21T22:36:43.982 INFO:tasks.workunit.client.0.smithi138.stdout: 1 68389 0.62 MB/sec execute 461 sec latency 128.682 ms 2024-01-21T22:36:44.982 INFO:tasks.workunit.client.0.smithi138.stdout: 1 68528 0.63 MB/sec execute 462 sec latency 105.554 ms 2024-01-21T22:36:45.982 INFO:tasks.workunit.client.0.smithi138.stdout: 1 68652 0.62 MB/sec execute 463 sec latency 125.003 ms 2024-01-21T22:36:46.982 INFO:tasks.workunit.client.0.smithi138.stdout: 1 68820 0.63 MB/sec execute 464 sec latency 283.431 ms 2024-01-21T22:36:47.982 INFO:tasks.workunit.client.0.smithi138.stdout: 1 68926 0.63 MB/sec execute 465 sec latency 186.887 ms 2024-01-21T22:36:48.982 INFO:tasks.workunit.client.0.smithi138.stdout: 1 69106 0.63 MB/sec execute 466 sec latency 97.117 ms 2024-01-21T22:36:49.983 INFO:tasks.workunit.client.0.smithi138.stdout: 1 69335 0.63 MB/sec execute 467 sec latency 139.613 ms 2024-01-21T22:36:50.983 INFO:tasks.workunit.client.0.smithi138.stdout: 1 69717 0.63 MB/sec execute 468 sec latency 163.539 ms 2024-01-21T22:36:51.983 INFO:tasks.workunit.client.0.smithi138.stdout: 1 69827 0.63 MB/sec execute 469 sec latency 118.768 ms 2024-01-21T22:36:52.983 INFO:tasks.workunit.client.0.smithi138.stdout: 1 69898 0.63 MB/sec execute 470 sec latency 195.064 ms 2024-01-21T22:36:53.983 INFO:tasks.workunit.client.0.smithi138.stdout: 1 70023 0.63 MB/sec execute 471 sec latency 120.925 ms 2024-01-21T22:36:54.983 INFO:tasks.workunit.client.0.smithi138.stdout: 1 70319 0.63 MB/sec execute 472 sec latency 84.974 ms 2024-01-21T22:36:55.983 INFO:tasks.workunit.client.0.smithi138.stdout: 1 70432 0.63 MB/sec execute 473 sec latency 104.284 ms 2024-01-21T22:36:56.984 INFO:tasks.workunit.client.0.smithi138.stdout: 1 70592 0.63 MB/sec execute 474 sec latency 462.803 ms 2024-01-21T22:36:57.984 INFO:tasks.workunit.client.0.smithi138.stdout: 1 70626 0.63 MB/sec execute 475 sec latency 493.282 ms 2024-01-21T22:36:58.984 INFO:tasks.workunit.client.0.smithi138.stdout: 1 70803 0.63 MB/sec execute 476 sec latency 120.279 ms 2024-01-21T22:36:59.984 INFO:tasks.workunit.client.0.smithi138.stdout: 1 71035 0.64 MB/sec execute 477 sec latency 101.015 ms 2024-01-21T22:37:00.984 INFO:tasks.workunit.client.0.smithi138.stdout: 1 71126 0.64 MB/sec execute 478 sec latency 87.339 ms 2024-01-21T22:37:01.984 INFO:tasks.workunit.client.0.smithi138.stdout: 1 71202 0.64 MB/sec execute 479 sec latency 119.904 ms 2024-01-21T22:37:02.984 INFO:tasks.workunit.client.0.smithi138.stdout: 1 71233 0.64 MB/sec execute 480 sec latency 192.242 ms 2024-01-21T22:37:03.984 INFO:tasks.workunit.client.0.smithi138.stdout: 1 71288 0.64 MB/sec execute 481 sec latency 103.139 ms 2024-01-21T22:37:04.985 INFO:tasks.workunit.client.0.smithi138.stdout: 1 71352 0.63 MB/sec execute 482 sec latency 100.229 ms 2024-01-21T22:37:05.985 INFO:tasks.workunit.client.0.smithi138.stdout: 1 71396 0.63 MB/sec execute 483 sec latency 105.387 ms 2024-01-21T22:37:06.985 INFO:tasks.workunit.client.0.smithi138.stdout: 1 71462 0.63 MB/sec execute 484 sec latency 96.499 ms 2024-01-21T22:37:07.985 INFO:tasks.workunit.client.0.smithi138.stdout: 1 71514 0.63 MB/sec execute 485 sec latency 198.329 ms 2024-01-21T22:37:08.075 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:37:08.083 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:37:08.985 INFO:tasks.workunit.client.0.smithi138.stdout: 1 71526 0.63 MB/sec execute 486 sec latency 473.544 ms 2024-01-21T22:37:09.985 INFO:tasks.workunit.client.0.smithi138.stdout: 1 71565 0.63 MB/sec execute 487 sec latency 291.092 ms 2024-01-21T22:37:10.985 INFO:tasks.workunit.client.0.smithi138.stdout: 1 71695 0.63 MB/sec execute 488 sec latency 142.860 ms 2024-01-21T22:37:11.986 INFO:tasks.workunit.client.0.smithi138.stdout: 1 71836 0.63 MB/sec execute 489 sec latency 141.630 ms 2024-01-21T22:37:12.986 INFO:tasks.workunit.client.0.smithi138.stdout: 1 71906 0.63 MB/sec execute 490 sec latency 168.276 ms 2024-01-21T22:37:13.986 INFO:tasks.workunit.client.0.smithi138.stdout: 1 72021 0.63 MB/sec execute 491 sec latency 181.990 ms 2024-01-21T22:37:14.986 INFO:tasks.workunit.client.0.smithi138.stdout: 1 72150 0.63 MB/sec execute 492 sec latency 94.473 ms 2024-01-21T22:37:15.986 INFO:tasks.workunit.client.0.smithi138.stdout: 1 72365 0.63 MB/sec execute 493 sec latency 159.502 ms 2024-01-21T22:37:16.986 INFO:tasks.workunit.client.0.smithi138.stdout: 1 72482 0.63 MB/sec execute 494 sec latency 223.976 ms 2024-01-21T22:37:17.986 INFO:tasks.workunit.client.0.smithi138.stdout: 1 72651 0.63 MB/sec execute 495 sec latency 105.634 ms 2024-01-21T22:37:18.986 INFO:tasks.workunit.client.0.smithi138.stdout: 1 72879 0.63 MB/sec execute 496 sec latency 150.862 ms 2024-01-21T22:37:19.986 INFO:tasks.workunit.client.0.smithi138.stdout: 1 73290 0.63 MB/sec execute 497 sec latency 201.978 ms 2024-01-21T22:37:20.987 INFO:tasks.workunit.client.0.smithi138.stdout: 1 73399 0.63 MB/sec execute 498 sec latency 123.064 ms 2024-01-21T22:37:21.987 INFO:tasks.workunit.client.0.smithi138.stdout: 1 73444 0.63 MB/sec execute 499 sec latency 369.308 ms 2024-01-21T22:37:22.987 INFO:tasks.workunit.client.0.smithi138.stdout: 1 73554 0.63 MB/sec execute 500 sec latency 235.443 ms 2024-01-21T22:37:23.987 INFO:tasks.workunit.client.0.smithi138.stdout: 1 73714 0.63 MB/sec execute 501 sec latency 152.274 ms 2024-01-21T22:37:24.987 INFO:tasks.workunit.client.0.smithi138.stdout: 1 73919 0.63 MB/sec execute 502 sec latency 86.179 ms 2024-01-21T22:37:25.987 INFO:tasks.workunit.client.0.smithi138.stdout: 1 74018 0.63 MB/sec execute 503 sec latency 136.047 ms 2024-01-21T22:37:26.987 INFO:tasks.workunit.client.0.smithi138.stdout: 1 74345 0.64 MB/sec execute 504 sec latency 116.858 ms 2024-01-21T22:37:27.987 INFO:tasks.workunit.client.0.smithi138.stdout: 1 74500 0.64 MB/sec execute 505 sec latency 131.522 ms 2024-01-21T22:37:28.988 INFO:tasks.workunit.client.0.smithi138.stdout: 1 74613 0.64 MB/sec execute 506 sec latency 180.139 ms 2024-01-21T22:37:29.988 INFO:tasks.workunit.client.0.smithi138.stdout: 1 74686 0.64 MB/sec execute 507 sec latency 184.876 ms 2024-01-21T22:37:30.988 INFO:tasks.workunit.client.0.smithi138.stdout: 1 74776 0.64 MB/sec execute 508 sec latency 110.131 ms 2024-01-21T22:37:31.988 INFO:tasks.workunit.client.0.smithi138.stdout: 1 74820 0.64 MB/sec execute 509 sec latency 116.480 ms 2024-01-21T22:37:32.988 INFO:tasks.workunit.client.0.smithi138.stdout: 1 74851 0.64 MB/sec execute 510 sec latency 364.721 ms 2024-01-21T22:37:33.988 INFO:tasks.workunit.client.0.smithi138.stdout: 1 74868 0.64 MB/sec execute 511 sec latency 296.056 ms 2024-01-21T22:37:34.988 INFO:tasks.workunit.client.0.smithi138.stdout: 1 74928 0.64 MB/sec execute 512 sec latency 118.698 ms 2024-01-21T22:37:35.988 INFO:tasks.workunit.client.0.smithi138.stdout: 1 74975 0.63 MB/sec execute 513 sec latency 101.646 ms 2024-01-21T22:37:36.033 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:13:20.226 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:37:36.041 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:13:20.234 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:37:36.079 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:13:20.337 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:37:36.083 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:13:20.341 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:37:36.097 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:13:20.355 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:37:36.098 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:13:20.357 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:37:36.160 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:13:20.418 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:37:36.194 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:13:20.387 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:37:36.195 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:13:20.452 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:37:36.989 INFO:tasks.workunit.client.0.smithi138.stdout: 1 75018 0.63 MB/sec execute 514 sec latency 102.755 ms 2024-01-21T22:37:37.989 INFO:tasks.workunit.client.0.smithi138.stdout: 1 75101 0.63 MB/sec execute 515 sec latency 110.331 ms 2024-01-21T22:37:38.196 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:37:38.204 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:37:38.989 INFO:tasks.workunit.client.0.smithi138.stdout: 1 75136 0.63 MB/sec execute 516 sec latency 188.977 ms 2024-01-21T22:37:39.989 INFO:tasks.workunit.client.0.smithi138.stdout: 1 75193 0.63 MB/sec execute 517 sec latency 363.225 ms 2024-01-21T22:37:40.989 INFO:tasks.workunit.client.0.smithi138.stdout: 1 75324 0.63 MB/sec execute 518 sec latency 192.650 ms 2024-01-21T22:37:41.989 INFO:tasks.workunit.client.0.smithi138.stdout: 1 75427 0.63 MB/sec execute 519 sec latency 219.642 ms 2024-01-21T22:37:42.990 INFO:tasks.workunit.client.0.smithi138.stdout: 1 75523 0.63 MB/sec execute 520 sec latency 279.029 ms 2024-01-21T22:37:43.990 INFO:tasks.workunit.client.0.smithi138.stdout: 1 75707 0.63 MB/sec execute 521 sec latency 89.880 ms 2024-01-21T22:37:44.990 INFO:tasks.workunit.client.0.smithi138.stdout: 1 75928 0.63 MB/sec execute 522 sec latency 93.500 ms 2024-01-21T22:37:45.990 INFO:tasks.workunit.client.0.smithi138.stdout: 1 75999 0.63 MB/sec execute 523 sec latency 114.599 ms 2024-01-21T22:37:46.990 INFO:tasks.workunit.client.0.smithi138.stdout: 1 76073 0.63 MB/sec execute 524 sec latency 126.977 ms 2024-01-21T22:37:47.990 INFO:tasks.workunit.client.0.smithi138.stdout: 1 76183 0.63 MB/sec execute 525 sec latency 124.684 ms 2024-01-21T22:37:48.990 INFO:tasks.workunit.client.0.smithi138.stdout: 1 76322 0.63 MB/sec execute 526 sec latency 118.652 ms 2024-01-21T22:37:49.990 INFO:tasks.workunit.client.0.smithi138.stdout: 1 76388 0.63 MB/sec execute 527 sec latency 292.756 ms 2024-01-21T22:37:50.991 INFO:tasks.workunit.client.0.smithi138.stdout: 1 76606 0.63 MB/sec execute 528 sec latency 251.583 ms 2024-01-21T22:37:51.991 INFO:tasks.workunit.client.0.smithi138.stdout: 1 76842 0.63 MB/sec execute 529 sec latency 171.026 ms 2024-01-21T22:37:52.991 INFO:tasks.workunit.client.0.smithi138.stdout: 1 76944 0.63 MB/sec execute 530 sec latency 212.262 ms 2024-01-21T22:37:53.991 INFO:tasks.workunit.client.0.smithi138.stdout: 1 77022 0.63 MB/sec execute 531 sec latency 117.968 ms 2024-01-21T22:37:54.991 INFO:tasks.workunit.client.0.smithi138.stdout: 1 77150 0.63 MB/sec execute 532 sec latency 113.588 ms 2024-01-21T22:37:55.991 INFO:tasks.workunit.client.0.smithi138.stdout: 1 77423 0.63 MB/sec execute 533 sec latency 123.558 ms 2024-01-21T22:37:56.992 INFO:tasks.workunit.client.0.smithi138.stdout: 1 77544 0.63 MB/sec execute 534 sec latency 96.837 ms 2024-01-21T22:37:57.992 INFO:tasks.workunit.client.0.smithi138.stdout: 1 77751 0.63 MB/sec execute 535 sec latency 157.700 ms 2024-01-21T22:37:58.992 INFO:tasks.workunit.client.0.smithi138.stdout: 1 77931 0.63 MB/sec execute 536 sec latency 149.397 ms 2024-01-21T22:37:59.992 INFO:tasks.workunit.client.0.smithi138.stdout: 1 78096 0.64 MB/sec execute 537 sec latency 109.383 ms 2024-01-21T22:38:00.992 INFO:tasks.workunit.client.0.smithi138.stdout: 1 78150 0.64 MB/sec execute 538 sec latency 709.667 ms 2024-01-21T22:38:01.992 INFO:tasks.workunit.client.0.smithi138.stdout: 1 78182 0.64 MB/sec execute 539 sec latency 972.263 ms 2024-01-21T22:38:02.992 INFO:tasks.workunit.client.0.smithi138.stdout: 1 78233 0.64 MB/sec execute 540 sec latency 144.842 ms 2024-01-21T22:38:03.993 INFO:tasks.workunit.client.0.smithi138.stdout: 1 78314 0.64 MB/sec execute 541 sec latency 125.739 ms 2024-01-21T22:38:04.993 INFO:tasks.workunit.client.0.smithi138.stdout: 1 78357 0.63 MB/sec execute 542 sec latency 125.753 ms 2024-01-21T22:38:05.993 INFO:tasks.workunit.client.0.smithi138.stdout: 1 78408 0.63 MB/sec execute 543 sec latency 84.692 ms 2024-01-21T22:38:06.993 INFO:tasks.workunit.client.0.smithi138.stdout: 1 78474 0.63 MB/sec execute 544 sec latency 118.415 ms 2024-01-21T22:38:07.993 INFO:tasks.workunit.client.0.smithi138.stdout: 1 78514 0.63 MB/sec execute 545 sec latency 100.022 ms 2024-01-21T22:38:08.328 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:38:08.336 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:38:08.993 INFO:tasks.workunit.client.0.smithi138.stdout: 1 78552 0.63 MB/sec execute 546 sec latency 170.083 ms 2024-01-21T22:38:09.994 INFO:tasks.workunit.client.0.smithi138.stdout: 1 78624 0.63 MB/sec execute 547 sec latency 210.329 ms 2024-01-21T22:38:10.994 INFO:tasks.workunit.client.0.smithi138.stdout: 1 78656 0.63 MB/sec execute 548 sec latency 243.579 ms 2024-01-21T22:38:11.994 INFO:tasks.workunit.client.0.smithi138.stdout: 1 78689 0.63 MB/sec execute 549 sec latency 352.035 ms 2024-01-21T22:38:12.994 INFO:tasks.workunit.client.0.smithi138.stdout: 1 78797 0.63 MB/sec execute 550 sec latency 186.877 ms 2024-01-21T22:38:13.994 INFO:tasks.workunit.client.0.smithi138.stdout: 1 78887 0.63 MB/sec execute 551 sec latency 193.466 ms 2024-01-21T22:38:14.994 INFO:tasks.workunit.client.0.smithi138.stdout: 1 79010 0.63 MB/sec execute 552 sec latency 116.612 ms 2024-01-21T22:38:15.994 INFO:tasks.workunit.client.0.smithi138.stdout: 1 79146 0.63 MB/sec execute 553 sec latency 117.675 ms 2024-01-21T22:38:16.994 INFO:tasks.workunit.client.0.smithi138.stdout: 1 79304 0.63 MB/sec execute 554 sec latency 92.931 ms 2024-01-21T22:38:17.995 INFO:tasks.workunit.client.0.smithi138.stdout: 1 79511 0.63 MB/sec execute 555 sec latency 123.648 ms 2024-01-21T22:38:18.995 INFO:tasks.workunit.client.0.smithi138.stdout: 1 79644 0.63 MB/sec execute 556 sec latency 127.471 ms 2024-01-21T22:38:19.995 INFO:tasks.workunit.client.0.smithi138.stdout: 1 79790 0.63 MB/sec execute 557 sec latency 126.683 ms 2024-01-21T22:38:20.995 INFO:tasks.workunit.client.0.smithi138.stdout: 1 80066 0.63 MB/sec execute 558 sec latency 126.071 ms 2024-01-21T22:38:21.995 INFO:tasks.workunit.client.0.smithi138.stdout: 1 80393 0.63 MB/sec execute 559 sec latency 167.073 ms 2024-01-21T22:38:22.995 INFO:tasks.workunit.client.0.smithi138.stdout: 1 80433 0.63 MB/sec execute 560 sec latency 368.941 ms 2024-01-21T22:38:23.995 INFO:tasks.workunit.client.0.smithi138.stdout: 1 80488 0.63 MB/sec execute 561 sec latency 371.505 ms 2024-01-21T22:38:24.995 INFO:tasks.workunit.client.0.smithi138.stdout: 1 80577 0.63 MB/sec execute 562 sec latency 120.507 ms 2024-01-21T22:38:25.996 INFO:tasks.workunit.client.0.smithi138.stdout: 1 80694 0.63 MB/sec execute 563 sec latency 148.053 ms 2024-01-21T22:38:26.996 INFO:tasks.workunit.client.0.smithi138.stdout: 1 81001 0.63 MB/sec execute 564 sec latency 70.847 ms 2024-01-21T22:38:27.996 INFO:tasks.workunit.client.0.smithi138.stdout: 1 81102 0.63 MB/sec execute 565 sec latency 147.227 ms 2024-01-21T22:38:28.996 INFO:tasks.workunit.client.0.smithi138.stdout: 1 81358 0.63 MB/sec execute 566 sec latency 158.299 ms 2024-01-21T22:38:29.996 INFO:tasks.workunit.client.0.smithi138.stdout: 1 81575 0.64 MB/sec execute 567 sec latency 124.541 ms 2024-01-21T22:38:30.996 INFO:tasks.workunit.client.0.smithi138.stdout: 1 81638 0.64 MB/sec execute 568 sec latency 178.107 ms 2024-01-21T22:38:31.996 INFO:tasks.workunit.client.0.smithi138.stdout: 1 81746 0.64 MB/sec execute 569 sec latency 161.330 ms 2024-01-21T22:38:32.997 INFO:tasks.workunit.client.0.smithi138.stdout: 1 81790 0.64 MB/sec execute 570 sec latency 182.077 ms 2024-01-21T22:38:33.997 INFO:tasks.workunit.client.0.smithi138.stdout: 1 81871 0.64 MB/sec execute 571 sec latency 124.737 ms 2024-01-21T22:38:34.997 INFO:tasks.workunit.client.0.smithi138.stdout: 1 81918 0.63 MB/sec execute 572 sec latency 116.519 ms 2024-01-21T22:38:35.997 INFO:tasks.workunit.client.0.smithi138.stdout: 1 81958 0.63 MB/sec execute 573 sec latency 171.499 ms 2024-01-21T22:38:36.997 INFO:tasks.workunit.client.0.smithi138.stdout: 1 82022 0.63 MB/sec execute 574 sec latency 99.487 ms 2024-01-21T22:38:37.999 INFO:tasks.workunit.client.0.smithi138.stdout: 1 82072 0.63 MB/sec execute 575 sec latency 96.264 ms 2024-01-21T22:38:38.505 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:38:38.513 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:38:38.997 INFO:tasks.workunit.client.0.smithi138.stdout: 1 82080 0.63 MB/sec execute 576 sec latency 590.262 ms 2024-01-21T22:38:39.997 INFO:tasks.workunit.client.0.smithi138.stdout: 1 82112 0.63 MB/sec execute 577 sec latency 619.000 ms 2024-01-21T22:38:40.998 INFO:tasks.workunit.client.0.smithi138.stdout: 1 82192 0.63 MB/sec execute 578 sec latency 181.200 ms 2024-01-21T22:38:41.998 INFO:tasks.workunit.client.0.smithi138.stdout: 1 82230 0.63 MB/sec execute 579 sec latency 265.121 ms 2024-01-21T22:38:42.998 INFO:tasks.workunit.client.0.smithi138.stdout: 1 82321 0.63 MB/sec execute 580 sec latency 275.281 ms 2024-01-21T22:38:43.998 INFO:tasks.workunit.client.0.smithi138.stdout: 1 82425 0.63 MB/sec execute 581 sec latency 148.523 ms 2024-01-21T22:38:44.998 INFO:tasks.workunit.client.0.smithi138.stdout: 1 82538 0.63 MB/sec execute 582 sec latency 130.048 ms 2024-01-21T22:38:45.998 INFO:tasks.workunit.client.0.smithi138.stdout: 1 82616 0.62 MB/sec execute 583 sec latency 129.630 ms 2024-01-21T22:38:46.999 INFO:tasks.workunit.client.0.smithi138.stdout: 1 82739 0.63 MB/sec execute 584 sec latency 129.992 ms 2024-01-21T22:38:47.999 INFO:tasks.workunit.client.0.smithi138.stdout: 1 82873 0.62 MB/sec execute 585 sec latency 108.672 ms 2024-01-21T22:38:48.999 INFO:tasks.workunit.client.0.smithi138.stdout: 1 83045 0.63 MB/sec execute 586 sec latency 104.205 ms 2024-01-21T22:38:49.999 INFO:tasks.workunit.client.0.smithi138.stdout: 1 83131 0.63 MB/sec execute 587 sec latency 117.326 ms 2024-01-21T22:38:50.999 INFO:tasks.workunit.client.0.smithi138.stdout: 1 83229 0.63 MB/sec execute 588 sec latency 101.972 ms 2024-01-21T22:38:51.999 INFO:tasks.workunit.client.0.smithi138.stdout: 1 83330 0.63 MB/sec execute 589 sec latency 128.044 ms 2024-01-21T22:38:52.999 INFO:tasks.workunit.client.0.smithi138.stdout: 1 83529 0.63 MB/sec execute 590 sec latency 221.623 ms 2024-01-21T22:38:53.999 INFO:tasks.workunit.client.0.smithi138.stdout: 1 83885 0.62 MB/sec execute 591 sec latency 140.641 ms 2024-01-21T22:38:54.999 INFO:tasks.workunit.client.0.smithi138.stdout: 1 84037 0.63 MB/sec execute 592 sec latency 110.581 ms 2024-01-21T22:38:56.000 INFO:tasks.workunit.client.0.smithi138.stdout: 1 84114 0.63 MB/sec execute 593 sec latency 124.033 ms 2024-01-21T22:38:57.000 INFO:tasks.workunit.client.0.smithi138.stdout: 1 84242 0.63 MB/sec execute 594 sec latency 125.167 ms 2024-01-21T22:38:58.000 INFO:tasks.workunit.client.0.smithi138.stdout: 1 84528 0.63 MB/sec execute 595 sec latency 130.752 ms 2024-01-21T22:38:59.000 INFO:tasks.workunit.client.0.smithi138.stdout: 1 84634 0.63 MB/sec execute 596 sec latency 114.416 ms 2024-01-21T22:39:00.000 INFO:tasks.workunit.client.0.smithi138.stdout: 1 84842 0.63 MB/sec execute 597 sec latency 167.798 ms 2024-01-21T22:39:01.000 INFO:tasks.workunit.client.0.smithi138.stdout: 1 85010 0.63 MB/sec execute 598 sec latency 103.652 ms 2024-01-21T22:39:02.000 INFO:tasks.workunit.client.0.smithi138.stdout: 1 85168 0.64 MB/sec execute 599 sec latency 133.313 ms 2024-01-21T22:39:03.001 INFO:tasks.workunit.client.0.smithi138.stdout: 1 cleanup 600 sec 2024-01-21T22:39:04.001 INFO:tasks.workunit.client.0.smithi138.stdout: 1 cleanup 601 sec 2024-01-21T22:39:05.001 INFO:tasks.workunit.client.0.smithi138.stdout: 1 cleanup 602 sec 2024-01-21T22:39:06.001 INFO:tasks.workunit.client.0.smithi138.stdout: 1 cleanup 603 sec 2024-01-21T22:39:07.001 INFO:tasks.workunit.client.0.smithi138.stdout: 1 cleanup 604 sec 2024-01-21T22:39:08.001 INFO:tasks.workunit.client.0.smithi138.stdout: 1 cleanup 605 sec 2024-01-21T22:39:08.629 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:39:08.637 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:39:09.001 INFO:tasks.workunit.client.0.smithi138.stdout: 1 cleanup 606 sec 2024-01-21T22:39:10.002 INFO:tasks.workunit.client.0.smithi138.stdout: 1 cleanup 607 sec 2024-01-21T22:39:10.036 INFO:tasks.workunit.client.0.smithi138.stdout: 0 cleanup 607 sec 2024-01-21T22:39:10.036 INFO:tasks.workunit.client.0.smithi138.stdout: 2024-01-21T22:39:10.037 INFO:tasks.workunit.client.0.smithi138.stdout: Operation Count AvgLat MaxLat 2024-01-21T22:39:10.037 INFO:tasks.workunit.client.0.smithi138.stdout: ---------------------------------------- 2024-01-21T22:39:10.037 INFO:tasks.workunit.client.0.smithi138.stdout: NTCreateX 12146 18.028 1567.841 2024-01-21T22:39:10.037 INFO:tasks.workunit.client.0.smithi138.stdout: Close 8964 0.011 0.341 2024-01-21T22:39:10.037 INFO:tasks.workunit.client.0.smithi138.stdout: Rename 517 86.484 604.365 2024-01-21T22:39:10.037 INFO:tasks.workunit.client.0.smithi138.stdout: Unlink 2441 49.369 493.256 2024-01-21T22:39:10.037 INFO:tasks.workunit.client.0.smithi138.stdout: Qpathinfo 11046 13.194 1326.765 2024-01-21T22:39:10.037 INFO:tasks.workunit.client.0.smithi138.stdout: Qfileinfo 1937 0.005 0.042 2024-01-21T22:39:10.037 INFO:tasks.workunit.client.0.smithi138.stdout: Qfsinfo 2028 12.349 214.775 2024-01-21T22:39:10.037 INFO:tasks.workunit.client.0.smithi138.stdout: Sfileinfo 1000 5.218 383.278 2024-01-21T22:39:10.037 INFO:tasks.workunit.client.0.smithi138.stdout: Find 4288 3.262 387.331 2024-01-21T22:39:10.037 INFO:tasks.workunit.client.0.smithi138.stdout: WriteX 6032 0.347 90.980 2024-01-21T22:39:10.038 INFO:tasks.workunit.client.0.smithi138.stdout: ReadX 19237 0.080 42.057 2024-01-21T22:39:10.038 INFO:tasks.workunit.client.0.smithi138.stdout: LockX 40 19.980 80.880 2024-01-21T22:39:10.038 INFO:tasks.workunit.client.0.smithi138.stdout: UnlockX 40 17.578 61.187 2024-01-21T22:39:10.038 INFO:tasks.workunit.client.0.smithi138.stdout: Flush 860 23.406 371.489 2024-01-21T22:39:10.038 INFO:tasks.workunit.client.0.smithi138.stdout: 2024-01-21T22:39:10.038 INFO:tasks.workunit.client.0.smithi138.stdout:Throughput 0.635718 MB/sec 1 clients 1 procs max_latency=1567.856 ms 2024-01-21T22:39:10.038 INFO:tasks.workunit.client.0.smithi138.stderr:+ dbench 10 2024-01-21T22:39:10.040 INFO:tasks.workunit.client.0.smithi138.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-01-21T22:39:10.040 INFO:tasks.workunit.client.0.smithi138.stdout: 2024-01-21T22:39:10.040 INFO:tasks.workunit.client.0.smithi138.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-01-21T22:39:10.043 INFO:tasks.workunit.client.0.smithi138.stdout:8 of 10 processes prepared for launch 0 sec 2024-01-21T22:39:10.143 INFO:tasks.workunit.client.0.smithi138.stdout:10 of 10 processes prepared for launch 0 sec 2024-01-21T22:39:10.144 INFO:tasks.workunit.client.0.smithi138.stdout:releasing clients 2024-01-21T22:39:11.144 INFO:tasks.workunit.client.0.smithi138.stdout: 3 15 5.24 MB/sec warmup 1 sec latency 999.573 ms 2024-01-21T22:39:12.144 INFO:tasks.workunit.client.0.smithi138.stdout: 6 36 7.34 MB/sec warmup 2 sec latency 1999.726 ms 2024-01-21T22:39:13.144 INFO:tasks.workunit.client.0.smithi138.stdout: 8 56 8.12 MB/sec warmup 3 sec latency 2999.853 ms 2024-01-21T22:39:14.144 INFO:tasks.workunit.client.0.smithi138.stdout: 10 80 8.84 MB/sec warmup 4 sec latency 1097.028 ms 2024-01-21T22:39:15.144 INFO:tasks.workunit.client.0.smithi138.stdout: 10 100 9.01 MB/sec warmup 5 sec latency 1022.485 ms 2024-01-21T22:39:16.145 INFO:tasks.workunit.client.0.smithi138.stdout: 10 115 8.64 MB/sec warmup 6 sec latency 1102.287 ms 2024-01-21T22:39:17.144 INFO:tasks.workunit.client.0.smithi138.stdout: 10 133 8.34 MB/sec warmup 7 sec latency 1053.618 ms 2024-01-21T22:39:18.145 INFO:tasks.workunit.client.0.smithi138.stdout: 10 144 7.76 MB/sec warmup 8 sec latency 953.981 ms 2024-01-21T22:39:19.145 INFO:tasks.workunit.client.0.smithi138.stdout: 10 152 7.18 MB/sec warmup 9 sec latency 1067.474 ms 2024-01-21T22:39:20.145 INFO:tasks.workunit.client.0.smithi138.stdout: 10 162 6.75 MB/sec warmup 10 sec latency 1289.093 ms 2024-01-21T22:39:21.145 INFO:tasks.workunit.client.0.smithi138.stdout: 10 172 6.40 MB/sec warmup 11 sec latency 1210.305 ms 2024-01-21T22:39:22.145 INFO:tasks.workunit.client.0.smithi138.stdout: 10 180 6.08 MB/sec warmup 12 sec latency 993.402 ms 2024-01-21T22:39:23.145 INFO:tasks.workunit.client.0.smithi138.stdout: 10 189 5.83 MB/sec warmup 13 sec latency 1193.307 ms 2024-01-21T22:39:24.145 INFO:tasks.workunit.client.0.smithi138.stdout: 10 196 5.56 MB/sec warmup 14 sec latency 1323.719 ms 2024-01-21T22:39:25.145 INFO:tasks.workunit.client.0.smithi138.stdout: 10 200 5.28 MB/sec warmup 15 sec latency 1951.978 ms 2024-01-21T22:39:26.146 INFO:tasks.workunit.client.0.smithi138.stdout: 10 209 5.13 MB/sec warmup 16 sec latency 2042.643 ms 2024-01-21T22:39:27.146 INFO:tasks.workunit.client.0.smithi138.stdout: 10 218 5.00 MB/sec warmup 17 sec latency 1286.403 ms 2024-01-21T22:39:28.146 INFO:tasks.workunit.client.0.smithi138.stdout: 10 227 4.88 MB/sec warmup 18 sec latency 1099.959 ms 2024-01-21T22:39:29.146 INFO:tasks.workunit.client.0.smithi138.stdout: 10 236 4.77 MB/sec warmup 19 sec latency 1125.608 ms 2024-01-21T22:39:30.146 INFO:tasks.workunit.client.0.smithi138.stdout: 10 236 4.53 MB/sec warmup 20 sec latency 1875.518 ms 2024-01-21T22:39:31.147 INFO:tasks.workunit.client.0.smithi138.stdout: 10 237 4.33 MB/sec warmup 21 sec latency 2875.666 ms 2024-01-21T22:39:32.147 INFO:tasks.workunit.client.0.smithi138.stdout: 10 248 4.30 MB/sec warmup 22 sec latency 2906.625 ms 2024-01-21T22:39:33.147 INFO:tasks.workunit.client.0.smithi138.stdout: 10 257 4.24 MB/sec warmup 23 sec latency 1149.814 ms 2024-01-21T22:39:34.147 INFO:tasks.workunit.client.0.smithi138.stdout: 10 262 4.12 MB/sec warmup 24 sec latency 1379.647 ms 2024-01-21T22:39:35.147 INFO:tasks.workunit.client.0.smithi138.stdout: 10 266 4.01 MB/sec warmup 25 sec latency 1975.183 ms 2024-01-21T22:39:36.147 INFO:tasks.workunit.client.0.smithi138.stdout: 10 266 3.86 MB/sec warmup 26 sec latency 2975.307 ms 2024-01-21T22:39:37.147 INFO:tasks.workunit.client.0.smithi138.stdout: 10 266 3.71 MB/sec warmup 27 sec latency 3975.439 ms 2024-01-21T22:39:38.147 INFO:tasks.workunit.client.0.smithi138.stdout: 10 270 3.63 MB/sec warmup 28 sec latency 4285.360 ms 2024-01-21T22:39:38.776 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:39:38.784 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:39:39.147 INFO:tasks.workunit.client.0.smithi138.stdout: 10 275 3.56 MB/sec warmup 29 sec latency 4272.050 ms 2024-01-21T22:39:40.148 INFO:tasks.workunit.client.0.smithi138.stdout: 10 275 3.44 MB/sec warmup 30 sec latency 2689.527 ms 2024-01-21T22:39:41.148 INFO:tasks.workunit.client.0.smithi138.stdout: 10 287 3.45 MB/sec warmup 31 sec latency 2739.039 ms 2024-01-21T22:39:42.148 INFO:tasks.workunit.client.0.smithi138.stdout: 10 295 3.41 MB/sec warmup 32 sec latency 925.362 ms 2024-01-21T22:39:43.148 INFO:tasks.workunit.client.0.smithi138.stdout: 10 307 3.42 MB/sec warmup 33 sec latency 1082.514 ms 2024-01-21T22:39:44.148 INFO:tasks.workunit.client.0.smithi138.stdout: 10 314 3.38 MB/sec warmup 34 sec latency 1172.416 ms 2024-01-21T22:39:45.148 INFO:tasks.workunit.client.0.smithi138.stdout: 10 321 3.35 MB/sec warmup 35 sec latency 1523.913 ms 2024-01-21T22:39:46.148 INFO:tasks.workunit.client.0.smithi138.stdout: 10 330 3.33 MB/sec warmup 36 sec latency 972.187 ms 2024-01-21T22:39:47.149 INFO:tasks.workunit.client.0.smithi138.stdout: 10 344 3.36 MB/sec warmup 37 sec latency 1039.414 ms 2024-01-21T22:39:48.149 INFO:tasks.workunit.client.0.smithi138.stdout: 10 354 3.34 MB/sec warmup 38 sec latency 1024.837 ms 2024-01-21T22:39:49.149 INFO:tasks.workunit.client.0.smithi138.stdout: 10 363 3.33 MB/sec warmup 39 sec latency 1257.104 ms 2024-01-21T22:39:50.149 INFO:tasks.workunit.client.0.smithi138.stdout: 10 371 3.31 MB/sec warmup 40 sec latency 1248.516 ms 2024-01-21T22:39:51.149 INFO:tasks.workunit.client.0.smithi138.stdout: 10 382 3.32 MB/sec warmup 41 sec latency 1013.149 ms 2024-01-21T22:39:52.150 INFO:tasks.workunit.client.0.smithi138.stdout: 10 390 3.30 MB/sec warmup 42 sec latency 923.429 ms 2024-01-21T22:39:53.149 INFO:tasks.workunit.client.0.smithi138.stdout: 10 401 3.31 MB/sec warmup 43 sec latency 1059.552 ms 2024-01-21T22:39:54.150 INFO:tasks.workunit.client.0.smithi138.stdout: 10 408 3.28 MB/sec warmup 44 sec latency 1058.464 ms 2024-01-21T22:39:55.150 INFO:tasks.workunit.client.0.smithi138.stdout: 10 415 3.26 MB/sec warmup 45 sec latency 1381.436 ms 2024-01-21T22:39:56.150 INFO:tasks.workunit.client.0.smithi138.stdout: 10 421 3.24 MB/sec warmup 46 sec latency 1501.413 ms 2024-01-21T22:39:57.150 INFO:tasks.workunit.client.0.smithi138.stdout: 10 431 3.23 MB/sec warmup 47 sec latency 1232.191 ms 2024-01-21T22:39:58.151 INFO:tasks.workunit.client.0.smithi138.stdout: 10 440 3.23 MB/sec warmup 48 sec latency 1079.485 ms 2024-01-21T22:39:59.150 INFO:tasks.workunit.client.0.smithi138.stdout: 10 448 3.21 MB/sec warmup 49 sec latency 1229.198 ms 2024-01-21T22:40:00.150 INFO:tasks.workunit.client.0.smithi138.stdout: 10 456 3.20 MB/sec warmup 50 sec latency 1140.432 ms 2024-01-21T22:40:01.151 INFO:tasks.workunit.client.0.smithi138.stdout: 10 464 3.19 MB/sec warmup 51 sec latency 1285.122 ms 2024-01-21T22:40:02.151 INFO:tasks.workunit.client.0.smithi138.stdout: 10 471 3.17 MB/sec warmup 52 sec latency 1123.262 ms 2024-01-21T22:40:03.151 INFO:tasks.workunit.client.0.smithi138.stdout: 10 478 3.16 MB/sec warmup 53 sec latency 1239.980 ms 2024-01-21T22:40:04.151 INFO:tasks.workunit.client.0.smithi138.stdout: 10 484 3.13 MB/sec warmup 54 sec latency 1374.706 ms 2024-01-21T22:40:05.151 INFO:tasks.workunit.client.0.smithi138.stdout: 10 493 3.12 MB/sec warmup 55 sec latency 2064.932 ms 2024-01-21T22:40:06.151 INFO:tasks.workunit.client.0.smithi138.stdout: 10 498 3.10 MB/sec warmup 56 sec latency 1586.851 ms 2024-01-21T22:40:07.151 INFO:tasks.workunit.client.0.smithi138.stdout: 10 504 3.08 MB/sec warmup 57 sec latency 1801.688 ms 2024-01-21T22:40:08.151 INFO:tasks.workunit.client.0.smithi138.stdout: 10 520 3.13 MB/sec warmup 58 sec latency 2084.348 ms 2024-01-21T22:40:08.915 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:40:08.923 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:40:09.152 INFO:tasks.workunit.client.0.smithi138.stdout: 10 542 3.18 MB/sec warmup 59 sec latency 1308.274 ms 2024-01-21T22:40:10.152 INFO:tasks.workunit.client.0.smithi138.stdout: 10 571 3.25 MB/sec warmup 60 sec latency 1240.295 ms 2024-01-21T22:40:11.152 INFO:tasks.workunit.client.0.smithi138.stdout: 10 590 3.26 MB/sec warmup 61 sec latency 1059.009 ms 2024-01-21T22:40:12.152 INFO:tasks.workunit.client.0.smithi138.stdout: 10 606 3.27 MB/sec warmup 62 sec latency 1205.344 ms 2024-01-21T22:40:13.152 INFO:tasks.workunit.client.0.smithi138.stdout: 10 635 3.29 MB/sec warmup 63 sec latency 1213.127 ms 2024-01-21T22:40:14.152 INFO:tasks.workunit.client.0.smithi138.stdout: 10 652 3.29 MB/sec warmup 64 sec latency 1499.673 ms 2024-01-21T22:40:15.152 INFO:tasks.workunit.client.0.smithi138.stdout: 10 669 3.25 MB/sec warmup 65 sec latency 1601.955 ms 2024-01-21T22:40:16.152 INFO:tasks.workunit.client.0.smithi138.stdout: 10 674 3.20 MB/sec warmup 66 sec latency 1759.309 ms 2024-01-21T22:40:17.153 INFO:tasks.workunit.client.0.smithi138.stdout: 10 677 3.15 MB/sec warmup 67 sec latency 2759.468 ms 2024-01-21T22:40:18.153 INFO:tasks.workunit.client.0.smithi138.stdout: 10 682 3.11 MB/sec warmup 68 sec latency 2996.531 ms 2024-01-21T22:40:19.153 INFO:tasks.workunit.client.0.smithi138.stdout: 10 694 3.07 MB/sec warmup 69 sec latency 3105.188 ms 2024-01-21T22:40:20.153 INFO:tasks.workunit.client.0.smithi138.stdout: 10 712 3.03 MB/sec warmup 70 sec latency 1739.275 ms 2024-01-21T22:40:21.153 INFO:tasks.workunit.client.0.smithi138.stdout: 10 719 2.99 MB/sec warmup 71 sec latency 2043.469 ms 2024-01-21T22:40:22.153 INFO:tasks.workunit.client.0.smithi138.stdout: 10 720 2.95 MB/sec warmup 72 sec latency 2912.268 ms 2024-01-21T22:40:23.154 INFO:tasks.workunit.client.0.smithi138.stdout: 10 727 2.91 MB/sec warmup 73 sec latency 3551.811 ms 2024-01-21T22:40:24.154 INFO:tasks.workunit.client.0.smithi138.stdout: 10 730 2.87 MB/sec warmup 74 sec latency 2558.070 ms 2024-01-21T22:40:25.154 INFO:tasks.workunit.client.0.smithi138.stdout: 10 736 2.84 MB/sec warmup 75 sec latency 2322.691 ms 2024-01-21T22:40:26.155 INFO:tasks.workunit.client.0.smithi138.stdout: 10 740 2.80 MB/sec warmup 76 sec latency 1914.248 ms 2024-01-21T22:40:27.155 INFO:tasks.workunit.client.0.smithi138.stdout: 10 740 2.76 MB/sec warmup 77 sec latency 1873.302 ms 2024-01-21T22:40:28.155 INFO:tasks.workunit.client.0.smithi138.stdout: 10 746 2.73 MB/sec warmup 78 sec latency 2404.718 ms 2024-01-21T22:40:29.155 INFO:tasks.workunit.client.0.smithi138.stdout: 10 764 2.71 MB/sec warmup 79 sec latency 1402.038 ms 2024-01-21T22:40:30.155 INFO:tasks.workunit.client.0.smithi138.stdout: 10 771 2.67 MB/sec warmup 80 sec latency 1991.772 ms 2024-01-21T22:40:31.155 INFO:tasks.workunit.client.0.smithi138.stdout: 10 783 2.64 MB/sec warmup 81 sec latency 2138.919 ms 2024-01-21T22:40:32.156 INFO:tasks.workunit.client.0.smithi138.stdout: 10 784 2.61 MB/sec warmup 82 sec latency 2388.211 ms 2024-01-21T22:40:33.156 INFO:tasks.workunit.client.0.smithi138.stdout: 10 796 2.58 MB/sec warmup 83 sec latency 2937.423 ms 2024-01-21T22:40:34.156 INFO:tasks.workunit.client.0.smithi138.stdout: 10 803 2.55 MB/sec warmup 84 sec latency 2833.101 ms 2024-01-21T22:40:35.156 INFO:tasks.workunit.client.0.smithi138.stdout: 10 807 2.52 MB/sec warmup 85 sec latency 1888.059 ms 2024-01-21T22:40:36.156 INFO:tasks.workunit.client.0.smithi138.stdout: 10 814 2.50 MB/sec warmup 86 sec latency 2032.414 ms 2024-01-21T22:40:37.156 INFO:tasks.workunit.client.0.smithi138.stdout: 10 816 2.47 MB/sec warmup 87 sec latency 2005.578 ms 2024-01-21T22:40:38.156 INFO:tasks.workunit.client.0.smithi138.stdout: 10 827 2.45 MB/sec warmup 88 sec latency 2290.710 ms 2024-01-21T22:40:39.070 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:40:39.078 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:40:39.156 INFO:tasks.workunit.client.0.smithi138.stdout: 10 839 2.42 MB/sec warmup 89 sec latency 1282.793 ms 2024-01-21T22:40:40.157 INFO:tasks.workunit.client.0.smithi138.stdout: 10 850 2.40 MB/sec warmup 90 sec latency 1639.388 ms 2024-01-21T22:40:41.157 INFO:tasks.workunit.client.0.smithi138.stdout: 10 861 2.37 MB/sec warmup 91 sec latency 2259.350 ms 2024-01-21T22:40:42.159 INFO:tasks.workunit.client.0.smithi138.stdout: 10 868 2.35 MB/sec warmup 92 sec latency 1511.482 ms 2024-01-21T22:40:43.157 INFO:tasks.workunit.client.0.smithi138.stdout: 10 874 2.33 MB/sec warmup 93 sec latency 1965.531 ms 2024-01-21T22:40:44.157 INFO:tasks.workunit.client.0.smithi138.stdout: 10 880 2.30 MB/sec warmup 94 sec latency 2569.400 ms 2024-01-21T22:40:45.157 INFO:tasks.workunit.client.0.smithi138.stdout: 10 891 2.28 MB/sec warmup 95 sec latency 1229.316 ms 2024-01-21T22:40:46.157 INFO:tasks.workunit.client.0.smithi138.stdout: 10 900 2.26 MB/sec warmup 96 sec latency 1316.874 ms 2024-01-21T22:40:47.157 INFO:tasks.workunit.client.0.smithi138.stdout: 10 905 2.24 MB/sec warmup 97 sec latency 1588.423 ms 2024-01-21T22:40:48.158 INFO:tasks.workunit.client.0.smithi138.stdout: 10 920 2.22 MB/sec warmup 98 sec latency 1696.196 ms 2024-01-21T22:40:49.158 INFO:tasks.workunit.client.0.smithi138.stdout: 10 926 2.20 MB/sec warmup 99 sec latency 1741.237 ms 2024-01-21T22:40:50.158 INFO:tasks.workunit.client.0.smithi138.stdout: 10 933 2.18 MB/sec warmup 100 sec latency 1289.426 ms 2024-01-21T22:40:51.158 INFO:tasks.workunit.client.0.smithi138.stdout: 10 941 2.16 MB/sec warmup 101 sec latency 1418.871 ms 2024-01-21T22:40:52.158 INFO:tasks.workunit.client.0.smithi138.stdout: 10 953 2.14 MB/sec warmup 102 sec latency 1248.435 ms 2024-01-21T22:40:53.158 INFO:tasks.workunit.client.0.smithi138.stdout: 10 964 2.12 MB/sec warmup 103 sec latency 1487.315 ms 2024-01-21T22:40:54.158 INFO:tasks.workunit.client.0.smithi138.stdout: 10 974 2.10 MB/sec warmup 104 sec latency 1198.339 ms 2024-01-21T22:40:55.158 INFO:tasks.workunit.client.0.smithi138.stdout: 10 982 2.10 MB/sec warmup 105 sec latency 1410.373 ms 2024-01-21T22:40:56.159 INFO:tasks.workunit.client.0.smithi138.stdout: 10 995 2.09 MB/sec warmup 106 sec latency 1529.828 ms 2024-01-21T22:40:57.159 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1007 2.08 MB/sec warmup 107 sec latency 1110.898 ms 2024-01-21T22:40:58.159 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1015 2.07 MB/sec warmup 108 sec latency 1041.401 ms 2024-01-21T22:40:59.159 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1028 2.07 MB/sec warmup 109 sec latency 974.809 ms 2024-01-21T22:41:00.159 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1042 2.07 MB/sec warmup 110 sec latency 965.559 ms 2024-01-21T22:41:01.159 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1055 2.05 MB/sec warmup 111 sec latency 1140.812 ms 2024-01-21T22:41:02.159 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1069 2.04 MB/sec warmup 112 sec latency 971.783 ms 2024-01-21T22:41:03.160 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1080 2.03 MB/sec warmup 113 sec latency 1047.986 ms 2024-01-21T22:41:04.160 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1098 2.01 MB/sec warmup 114 sec latency 974.356 ms 2024-01-21T22:41:04.490 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:16:48.683 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:41:04.492 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:16:48.750 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:41:04.494 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:16:48.687 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:41:04.495 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:16:48.753 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:41:04.541 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:16:48.799 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:41:04.545 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:16:48.803 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:41:04.614 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:16:48.872 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:41:04.655 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:16:48.913 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:41:04.796 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:16:48.989 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:41:05.160 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1108 2.00 MB/sec warmup 115 sec latency 1505.311 ms 2024-01-21T22:41:06.160 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1120 1.98 MB/sec warmup 116 sec latency 1829.137 ms 2024-01-21T22:41:07.160 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1128 1.96 MB/sec warmup 117 sec latency 1306.019 ms 2024-01-21T22:41:08.160 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1139 1.95 MB/sec warmup 118 sec latency 1331.965 ms 2024-01-21T22:41:09.160 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1152 1.93 MB/sec warmup 119 sec latency 1316.908 ms 2024-01-21T22:41:09.190 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:41:09.198 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:41:11.161 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1177 0.11 MB/sec execute 1 sec latency 1182.940 ms 2024-01-21T22:41:12.161 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1184 0.08 MB/sec execute 2 sec latency 1342.143 ms 2024-01-21T22:41:13.161 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1192 0.07 MB/sec execute 3 sec latency 1271.677 ms 2024-01-21T22:41:14.161 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1203 0.08 MB/sec execute 4 sec latency 1094.666 ms 2024-01-21T22:41:15.161 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1214 0.09 MB/sec execute 5 sec latency 1105.382 ms 2024-01-21T22:41:16.162 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1221 0.09 MB/sec execute 6 sec latency 1098.771 ms 2024-01-21T22:41:17.161 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1240 0.50 MB/sec execute 7 sec latency 1503.009 ms 2024-01-21T22:41:18.162 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1240 0.44 MB/sec execute 8 sec latency 1944.243 ms 2024-01-21T22:41:19.162 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1240 0.39 MB/sec execute 9 sec latency 2944.376 ms 2024-01-21T22:41:20.162 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1247 0.36 MB/sec execute 10 sec latency 3440.784 ms 2024-01-21T22:41:21.162 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1261 0.37 MB/sec execute 11 sec latency 3475.357 ms 2024-01-21T22:41:22.162 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1277 0.63 MB/sec execute 12 sec latency 1662.255 ms 2024-01-21T22:41:23.162 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1284 0.73 MB/sec execute 13 sec latency 2528.945 ms 2024-01-21T22:41:24.162 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1298 0.69 MB/sec execute 14 sec latency 2463.208 ms 2024-01-21T22:41:25.162 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1307 0.65 MB/sec execute 15 sec latency 1189.086 ms 2024-01-21T22:41:26.163 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1321 0.72 MB/sec execute 16 sec latency 1600.752 ms 2024-01-21T22:41:27.163 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1334 0.86 MB/sec execute 17 sec latency 2124.021 ms 2024-01-21T22:41:28.163 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1352 1.02 MB/sec execute 18 sec latency 1810.657 ms 2024-01-21T22:41:29.163 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1366 1.04 MB/sec execute 19 sec latency 2229.596 ms 2024-01-21T22:41:30.163 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1380 1.14 MB/sec execute 20 sec latency 1854.512 ms 2024-01-21T22:41:31.163 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1394 1.20 MB/sec execute 21 sec latency 2202.003 ms 2024-01-21T22:41:32.164 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1412 1.28 MB/sec execute 22 sec latency 2552.945 ms 2024-01-21T22:41:33.164 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1428 1.31 MB/sec execute 23 sec latency 1812.055 ms 2024-01-21T22:41:34.164 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1439 1.26 MB/sec execute 24 sec latency 1327.353 ms 2024-01-21T22:41:35.164 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1450 1.21 MB/sec execute 25 sec latency 1237.662 ms 2024-01-21T22:41:36.164 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1458 1.16 MB/sec execute 26 sec latency 1487.831 ms 2024-01-21T22:41:37.164 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1469 1.12 MB/sec execute 27 sec latency 1084.869 ms 2024-01-21T22:41:38.164 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1477 1.08 MB/sec execute 28 sec latency 1344.794 ms 2024-01-21T22:41:39.164 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1485 1.05 MB/sec execute 29 sec latency 2086.005 ms 2024-01-21T22:41:39.331 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:41:39.339 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:41:40.164 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1494 1.01 MB/sec execute 30 sec latency 2070.882 ms 2024-01-21T22:41:41.165 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1508 0.99 MB/sec execute 31 sec latency 1814.841 ms 2024-01-21T22:41:42.165 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1521 0.96 MB/sec execute 32 sec latency 1050.907 ms 2024-01-21T22:41:43.165 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1527 0.93 MB/sec execute 33 sec latency 1564.001 ms 2024-01-21T22:41:44.165 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1539 0.91 MB/sec execute 34 sec latency 1921.537 ms 2024-01-21T22:41:45.165 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1548 0.88 MB/sec execute 35 sec latency 2275.686 ms 2024-01-21T22:41:46.165 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1562 0.86 MB/sec execute 36 sec latency 1715.567 ms 2024-01-21T22:41:47.165 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1570 0.84 MB/sec execute 37 sec latency 1540.045 ms 2024-01-21T22:41:48.165 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1582 0.82 MB/sec execute 38 sec latency 2087.493 ms 2024-01-21T22:41:48.721 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:17:32.915 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:41:48.732 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:17:32.926 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:41:48.775 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:17:33.032 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:41:48.777 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:17:33.035 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:41:48.808 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:17:33.066 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:41:48.823 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:17:33.081 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:41:48.825 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:17:33.083 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:41:48.833 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:17:33.027 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:41:48.855 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:17:33.111 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:41:49.165 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1584 0.80 MB/sec execute 39 sec latency 3087.580 ms 2024-01-21T22:41:50.166 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1593 0.78 MB/sec execute 40 sec latency 3109.677 ms 2024-01-21T22:41:51.166 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1609 0.77 MB/sec execute 41 sec latency 2923.695 ms 2024-01-21T22:41:52.166 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1621 0.75 MB/sec execute 42 sec latency 1415.163 ms 2024-01-21T22:41:53.166 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1633 0.74 MB/sec execute 43 sec latency 1847.104 ms 2024-01-21T22:41:54.166 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1643 0.72 MB/sec execute 44 sec latency 2374.332 ms 2024-01-21T22:41:55.166 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1648 0.71 MB/sec execute 45 sec latency 2131.893 ms 2024-01-21T22:41:56.167 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1665 0.69 MB/sec execute 46 sec latency 2420.515 ms 2024-01-21T22:41:57.167 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1680 0.68 MB/sec execute 47 sec latency 1925.256 ms 2024-01-21T22:41:58.167 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1689 0.67 MB/sec execute 48 sec latency 1840.323 ms 2024-01-21T22:41:59.167 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1707 0.66 MB/sec execute 49 sec latency 1735.586 ms 2024-01-21T22:42:00.167 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1719 0.65 MB/sec execute 50 sec latency 1782.506 ms 2024-01-21T22:42:01.167 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1739 0.64 MB/sec execute 51 sec latency 1793.227 ms 2024-01-21T22:42:02.167 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1760 0.63 MB/sec execute 52 sec latency 1412.011 ms 2024-01-21T22:42:03.168 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1779 0.62 MB/sec execute 53 sec latency 1089.161 ms 2024-01-21T22:42:04.168 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1795 0.61 MB/sec execute 54 sec latency 1222.096 ms 2024-01-21T22:42:05.168 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1813 0.60 MB/sec execute 55 sec latency 1456.446 ms 2024-01-21T22:42:06.168 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1838 0.59 MB/sec execute 56 sec latency 1434.571 ms 2024-01-21T22:42:07.168 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1854 0.59 MB/sec execute 57 sec latency 1924.036 ms 2024-01-21T22:42:08.168 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1882 0.58 MB/sec execute 58 sec latency 2793.323 ms 2024-01-21T22:42:09.168 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1924 0.58 MB/sec execute 59 sec latency 1220.759 ms 2024-01-21T22:42:09.469 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:42:09.478 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:42:10.168 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1953 0.58 MB/sec execute 60 sec latency 1384.584 ms 2024-01-21T22:42:11.169 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1982 0.57 MB/sec execute 61 sec latency 1356.805 ms 2024-01-21T22:42:12.169 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2007 0.57 MB/sec execute 62 sec latency 1193.557 ms 2024-01-21T22:42:13.169 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2038 0.56 MB/sec execute 63 sec latency 1198.645 ms 2024-01-21T22:42:14.169 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2068 0.60 MB/sec execute 64 sec latency 1124.882 ms 2024-01-21T22:42:15.169 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2068 0.59 MB/sec execute 65 sec latency 2125.027 ms 2024-01-21T22:42:16.170 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2074 0.59 MB/sec execute 66 sec latency 2556.172 ms 2024-01-21T22:42:17.170 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2107 0.60 MB/sec execute 67 sec latency 3059.049 ms 2024-01-21T22:42:18.170 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2112 0.59 MB/sec execute 68 sec latency 4059.188 ms 2024-01-21T22:42:19.170 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2132 0.58 MB/sec execute 69 sec latency 4133.236 ms 2024-01-21T22:42:20.170 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2156 0.58 MB/sec execute 70 sec latency 835.757 ms 2024-01-21T22:42:21.170 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2185 0.58 MB/sec execute 71 sec latency 1043.868 ms 2024-01-21T22:42:22.170 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2215 0.57 MB/sec execute 72 sec latency 760.018 ms 2024-01-21T22:42:23.170 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2236 0.60 MB/sec execute 73 sec latency 1131.995 ms 2024-01-21T22:42:24.171 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2259 0.59 MB/sec execute 74 sec latency 1315.944 ms 2024-01-21T22:42:25.171 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2282 0.60 MB/sec execute 75 sec latency 1178.240 ms 2024-01-21T22:42:26.171 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2286 0.59 MB/sec execute 76 sec latency 2044.092 ms 2024-01-21T22:42:27.171 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2296 0.59 MB/sec execute 77 sec latency 2596.370 ms 2024-01-21T22:42:28.171 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2302 0.58 MB/sec execute 78 sec latency 2207.778 ms 2024-01-21T22:42:29.171 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2319 0.58 MB/sec execute 79 sec latency 2570.849 ms 2024-01-21T22:42:30.171 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2334 0.58 MB/sec execute 80 sec latency 1022.672 ms 2024-01-21T22:42:31.171 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2349 0.58 MB/sec execute 81 sec latency 1014.524 ms 2024-01-21T22:42:32.172 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2372 0.60 MB/sec execute 82 sec latency 1079.407 ms 2024-01-21T22:42:33.172 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2379 0.60 MB/sec execute 83 sec latency 1077.071 ms 2024-01-21T22:42:34.172 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2387 0.59 MB/sec execute 84 sec latency 1305.043 ms 2024-01-21T22:42:35.172 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2394 0.58 MB/sec execute 85 sec latency 920.092 ms 2024-01-21T22:42:36.172 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2402 0.58 MB/sec execute 86 sec latency 1215.002 ms 2024-01-21T22:42:37.172 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2408 0.58 MB/sec execute 87 sec latency 1271.714 ms 2024-01-21T22:42:38.172 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2413 0.57 MB/sec execute 88 sec latency 1143.748 ms 2024-01-21T22:42:39.172 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2421 0.57 MB/sec execute 89 sec latency 1284.923 ms 2024-01-21T22:42:39.792 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:42:39.800 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:42:40.172 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2429 0.58 MB/sec execute 90 sec latency 1203.013 ms 2024-01-21T22:42:41.173 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2440 0.58 MB/sec execute 91 sec latency 997.542 ms 2024-01-21T22:42:42.173 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2447 0.58 MB/sec execute 92 sec latency 1105.166 ms 2024-01-21T22:42:43.174 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2453 0.57 MB/sec execute 93 sec latency 1759.713 ms 2024-01-21T22:42:44.173 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2461 0.57 MB/sec execute 94 sec latency 1295.312 ms 2024-01-21T22:42:45.173 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2475 0.57 MB/sec execute 95 sec latency 1103.835 ms 2024-01-21T22:42:46.173 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2492 0.57 MB/sec execute 96 sec latency 999.143 ms 2024-01-21T22:42:47.174 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2500 0.57 MB/sec execute 97 sec latency 1356.812 ms 2024-01-21T22:42:48.174 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2507 0.57 MB/sec execute 98 sec latency 1009.663 ms 2024-01-21T22:42:49.174 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2514 0.57 MB/sec execute 99 sec latency 1367.641 ms 2024-01-21T22:42:50.174 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2524 0.57 MB/sec execute 100 sec latency 1434.976 ms 2024-01-21T22:42:51.174 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2526 0.56 MB/sec execute 101 sec latency 1913.706 ms 2024-01-21T22:42:52.174 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2526 0.56 MB/sec execute 102 sec latency 2913.841 ms 2024-01-21T22:42:53.174 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2527 0.55 MB/sec execute 103 sec latency 3759.741 ms 2024-01-21T22:42:54.174 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2531 0.55 MB/sec execute 104 sec latency 3582.692 ms 2024-01-21T22:42:55.175 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2545 0.55 MB/sec execute 105 sec latency 1765.401 ms 2024-01-21T22:42:56.175 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2559 0.57 MB/sec execute 106 sec latency 1297.423 ms 2024-01-21T22:42:57.175 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2581 0.60 MB/sec execute 107 sec latency 1613.063 ms 2024-01-21T22:42:58.175 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2595 0.60 MB/sec execute 108 sec latency 1637.251 ms 2024-01-21T22:42:59.175 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2615 0.62 MB/sec execute 109 sec latency 1074.969 ms 2024-01-21T22:43:00.175 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2634 0.65 MB/sec execute 110 sec latency 1112.093 ms 2024-01-21T22:43:01.176 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2643 0.64 MB/sec execute 111 sec latency 1521.361 ms 2024-01-21T22:43:02.176 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2648 0.64 MB/sec execute 112 sec latency 1491.419 ms 2024-01-21T22:43:03.176 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2669 0.66 MB/sec execute 113 sec latency 1476.807 ms 2024-01-21T22:43:04.176 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2696 0.68 MB/sec execute 114 sec latency 1377.870 ms 2024-01-21T22:43:05.176 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2715 0.68 MB/sec execute 115 sec latency 976.549 ms 2024-01-21T22:43:06.176 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2745 0.68 MB/sec execute 116 sec latency 1022.076 ms 2024-01-21T22:43:07.176 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2753 0.69 MB/sec execute 117 sec latency 1106.688 ms 2024-01-21T22:43:08.176 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2771 0.73 MB/sec execute 118 sec latency 1855.938 ms 2024-01-21T22:43:09.177 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2793 0.74 MB/sec execute 119 sec latency 2165.878 ms 2024-01-21T22:43:09.938 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:43:09.946 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:43:10.177 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2802 0.73 MB/sec execute 120 sec latency 911.048 ms 2024-01-21T22:43:11.177 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2811 0.74 MB/sec execute 121 sec latency 1073.443 ms 2024-01-21T22:43:12.177 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2814 0.74 MB/sec execute 122 sec latency 1622.033 ms 2024-01-21T22:43:13.177 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2834 0.76 MB/sec execute 123 sec latency 2622.045 ms 2024-01-21T22:43:14.177 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2840 0.76 MB/sec execute 124 sec latency 2827.645 ms 2024-01-21T22:43:15.177 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2847 0.75 MB/sec execute 125 sec latency 1143.337 ms 2024-01-21T22:43:16.178 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2861 0.75 MB/sec execute 126 sec latency 1167.893 ms 2024-01-21T22:43:17.178 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2884 0.76 MB/sec execute 127 sec latency 1463.790 ms 2024-01-21T22:43:18.178 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2896 0.77 MB/sec execute 128 sec latency 1400.620 ms 2024-01-21T22:43:19.178 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2911 0.78 MB/sec execute 129 sec latency 1748.279 ms 2024-01-21T22:43:20.178 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2921 0.77 MB/sec execute 130 sec latency 981.145 ms 2024-01-21T22:43:21.178 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2940 0.78 MB/sec execute 131 sec latency 773.760 ms 2024-01-21T22:43:22.178 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2948 0.77 MB/sec execute 132 sec latency 885.668 ms 2024-01-21T22:43:23.179 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2970 0.78 MB/sec execute 133 sec latency 1499.351 ms 2024-01-21T22:43:24.179 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2979 0.77 MB/sec execute 134 sec latency 1233.386 ms 2024-01-21T22:43:25.179 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2995 0.77 MB/sec execute 135 sec latency 1688.487 ms 2024-01-21T22:43:26.179 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3016 0.77 MB/sec execute 136 sec latency 1636.454 ms 2024-01-21T22:43:27.179 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3039 0.78 MB/sec execute 137 sec latency 1018.967 ms 2024-01-21T22:43:28.179 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3071 0.78 MB/sec execute 138 sec latency 1331.957 ms 2024-01-21T22:43:29.179 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3075 0.77 MB/sec execute 139 sec latency 2015.531 ms 2024-01-21T22:43:30.179 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3083 0.77 MB/sec execute 140 sec latency 1729.006 ms 2024-01-21T22:43:31.180 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3095 0.78 MB/sec execute 141 sec latency 1074.155 ms 2024-01-21T22:43:32.180 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3122 0.78 MB/sec execute 142 sec latency 996.494 ms 2024-01-21T22:43:33.180 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3129 0.77 MB/sec execute 143 sec latency 1480.234 ms 2024-01-21T22:43:34.180 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3158 0.78 MB/sec execute 144 sec latency 1014.145 ms 2024-01-21T22:43:35.180 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3165 0.78 MB/sec execute 145 sec latency 1455.918 ms 2024-01-21T22:43:36.180 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3207 0.80 MB/sec execute 146 sec latency 1512.818 ms 2024-01-21T22:43:37.180 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3219 0.79 MB/sec execute 147 sec latency 1225.122 ms 2024-01-21T22:43:38.183 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3236 0.79 MB/sec execute 148 sec latency 1246.648 ms 2024-01-21T22:43:39.181 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3242 0.78 MB/sec execute 149 sec latency 1688.774 ms 2024-01-21T22:43:40.070 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:43:40.078 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:43:40.181 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3245 0.78 MB/sec execute 150 sec latency 1941.110 ms 2024-01-21T22:43:41.181 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3282 0.80 MB/sec execute 151 sec latency 1601.069 ms 2024-01-21T22:43:42.181 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3308 0.81 MB/sec execute 152 sec latency 997.112 ms 2024-01-21T22:43:43.181 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3322 0.81 MB/sec execute 153 sec latency 1414.854 ms 2024-01-21T22:43:44.181 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3334 0.81 MB/sec execute 154 sec latency 1151.075 ms 2024-01-21T22:43:45.181 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3353 0.82 MB/sec execute 155 sec latency 913.460 ms 2024-01-21T22:43:46.182 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3363 0.82 MB/sec execute 156 sec latency 1567.929 ms 2024-01-21T22:43:47.183 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3403 0.87 MB/sec execute 157 sec latency 1103.175 ms 2024-01-21T22:43:48.182 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3414 0.87 MB/sec execute 158 sec latency 1289.195 ms 2024-01-21T22:43:49.182 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3416 0.87 MB/sec execute 159 sec latency 1534.043 ms 2024-01-21T22:43:50.182 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3416 0.86 MB/sec execute 160 sec latency 2534.218 ms 2024-01-21T22:43:51.182 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3422 0.86 MB/sec execute 161 sec latency 2883.812 ms 2024-01-21T22:43:52.182 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3426 0.85 MB/sec execute 162 sec latency 1650.490 ms 2024-01-21T22:43:53.183 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3442 0.85 MB/sec execute 163 sec latency 2106.317 ms 2024-01-21T22:43:54.183 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3449 0.85 MB/sec execute 164 sec latency 2193.710 ms 2024-01-21T22:43:55.183 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3456 0.85 MB/sec execute 165 sec latency 2041.209 ms 2024-01-21T22:43:56.183 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3471 0.86 MB/sec execute 166 sec latency 1650.081 ms 2024-01-21T22:43:57.183 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3481 0.86 MB/sec execute 167 sec latency 1593.898 ms 2024-01-21T22:43:58.183 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3495 0.86 MB/sec execute 168 sec latency 1453.731 ms 2024-01-21T22:43:59.183 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3499 0.86 MB/sec execute 169 sec latency 2103.089 ms 2024-01-21T22:44:00.183 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3516 0.87 MB/sec execute 170 sec latency 1765.184 ms 2024-01-21T22:44:01.184 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3523 0.87 MB/sec execute 171 sec latency 994.406 ms 2024-01-21T22:44:02.184 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3532 0.86 MB/sec execute 172 sec latency 1364.195 ms 2024-01-21T22:44:03.184 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3555 0.88 MB/sec execute 173 sec latency 1405.023 ms 2024-01-21T22:44:04.184 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3563 0.87 MB/sec execute 174 sec latency 1502.844 ms 2024-01-21T22:44:05.184 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3569 0.87 MB/sec execute 175 sec latency 1503.372 ms 2024-01-21T22:44:06.184 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3576 0.86 MB/sec execute 176 sec latency 1157.624 ms 2024-01-21T22:44:07.184 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3596 0.88 MB/sec execute 177 sec latency 1365.534 ms 2024-01-21T22:44:08.185 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3601 0.87 MB/sec execute 178 sec latency 2063.756 ms 2024-01-21T22:44:09.185 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3606 0.87 MB/sec execute 179 sec latency 2327.548 ms 2024-01-21T22:44:10.185 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3614 0.86 MB/sec execute 180 sec latency 2363.106 ms 2024-01-21T22:44:10.231 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:44:10.239 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:44:11.185 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3620 0.86 MB/sec execute 181 sec latency 1062.908 ms 2024-01-21T22:44:12.185 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3631 0.86 MB/sec execute 182 sec latency 1002.265 ms 2024-01-21T22:44:13.185 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3637 0.85 MB/sec execute 183 sec latency 1433.081 ms 2024-01-21T22:44:14.185 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3650 0.85 MB/sec execute 184 sec latency 1551.266 ms 2024-01-21T22:44:15.185 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3654 0.85 MB/sec execute 185 sec latency 1314.049 ms 2024-01-21T22:44:16.186 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3660 0.84 MB/sec execute 186 sec latency 1251.989 ms 2024-01-21T22:44:17.186 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3675 0.84 MB/sec execute 187 sec latency 1318.462 ms 2024-01-21T22:44:18.186 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3679 0.83 MB/sec execute 188 sec latency 1445.424 ms 2024-01-21T22:44:19.186 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3695 0.84 MB/sec execute 189 sec latency 1979.030 ms 2024-01-21T22:44:20.186 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3706 0.85 MB/sec execute 190 sec latency 2063.087 ms 2024-01-21T22:44:21.187 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3710 0.84 MB/sec execute 191 sec latency 1391.973 ms 2024-01-21T22:44:22.187 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3721 0.84 MB/sec execute 192 sec latency 1442.391 ms 2024-01-21T22:44:23.187 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3730 0.85 MB/sec execute 193 sec latency 1578.380 ms 2024-01-21T22:44:24.187 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3735 0.84 MB/sec execute 194 sec latency 1627.974 ms 2024-01-21T22:44:25.187 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3741 0.84 MB/sec execute 195 sec latency 1335.382 ms 2024-01-21T22:44:26.187 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3745 0.83 MB/sec execute 196 sec latency 1170.610 ms 2024-01-21T22:44:27.187 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3753 0.83 MB/sec execute 197 sec latency 1473.366 ms 2024-01-21T22:44:28.188 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3760 0.84 MB/sec execute 198 sec latency 1503.356 ms 2024-01-21T22:44:29.188 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3763 0.83 MB/sec execute 199 sec latency 1650.476 ms 2024-01-21T22:44:30.188 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3768 0.83 MB/sec execute 200 sec latency 1740.997 ms 2024-01-21T22:44:31.188 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3770 0.82 MB/sec execute 201 sec latency 2093.414 ms 2024-01-21T22:44:32.188 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3774 0.82 MB/sec execute 202 sec latency 1496.061 ms 2024-01-21T22:44:33.188 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3777 0.82 MB/sec execute 203 sec latency 1406.745 ms 2024-01-21T22:44:34.188 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3780 0.81 MB/sec execute 204 sec latency 1382.380 ms 2024-01-21T22:44:35.188 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3783 0.81 MB/sec execute 205 sec latency 1889.490 ms 2024-01-21T22:44:36.189 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3788 0.80 MB/sec execute 206 sec latency 1420.295 ms 2024-01-21T22:44:37.189 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3791 0.80 MB/sec execute 207 sec latency 1266.111 ms 2024-01-21T22:44:38.189 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3797 0.80 MB/sec execute 208 sec latency 1398.515 ms 2024-01-21T22:44:39.189 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3797 0.80 MB/sec execute 209 sec latency 2207.240 ms 2024-01-21T22:44:40.189 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3799 0.79 MB/sec execute 210 sec latency 3150.138 ms 2024-01-21T22:44:40.621 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:44:40.629 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:44:41.189 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3802 0.79 MB/sec execute 211 sec latency 4150.263 ms 2024-01-21T22:44:42.189 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3806 0.79 MB/sec execute 212 sec latency 4199.553 ms 2024-01-21T22:44:43.189 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3809 0.78 MB/sec execute 213 sec latency 1612.045 ms 2024-01-21T22:44:44.190 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3816 0.78 MB/sec execute 214 sec latency 1266.744 ms 2024-01-21T22:44:45.190 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3823 0.78 MB/sec execute 215 sec latency 1194.566 ms 2024-01-21T22:44:46.190 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3825 0.78 MB/sec execute 216 sec latency 1141.302 ms 2024-01-21T22:44:47.190 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3831 0.77 MB/sec execute 217 sec latency 1236.979 ms 2024-01-21T22:44:48.190 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3835 0.77 MB/sec execute 218 sec latency 1313.223 ms 2024-01-21T22:44:49.190 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3839 0.76 MB/sec execute 219 sec latency 1574.647 ms 2024-01-21T22:44:50.190 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3842 0.76 MB/sec execute 220 sec latency 1402.257 ms 2024-01-21T22:44:51.190 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3847 0.76 MB/sec execute 221 sec latency 1391.340 ms 2024-01-21T22:44:52.191 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3852 0.75 MB/sec execute 222 sec latency 1290.548 ms 2024-01-21T22:44:53.191 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3854 0.75 MB/sec execute 223 sec latency 1078.294 ms 2024-01-21T22:44:54.191 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3860 0.75 MB/sec execute 224 sec latency 1539.834 ms 2024-01-21T22:44:55.191 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3870 0.75 MB/sec execute 225 sec latency 1522.166 ms 2024-01-21T22:44:56.191 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3875 0.75 MB/sec execute 226 sec latency 1043.098 ms 2024-01-21T22:44:57.191 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3882 0.75 MB/sec execute 227 sec latency 1374.130 ms 2024-01-21T22:44:58.191 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3885 0.74 MB/sec execute 228 sec latency 1431.393 ms 2024-01-21T22:44:59.191 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3890 0.74 MB/sec execute 229 sec latency 1540.362 ms 2024-01-21T22:45:00.192 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3893 0.74 MB/sec execute 230 sec latency 1633.120 ms 2024-01-21T22:45:01.192 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3896 0.73 MB/sec execute 231 sec latency 1632.331 ms 2024-01-21T22:45:02.192 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3900 0.73 MB/sec execute 232 sec latency 1775.655 ms 2024-01-21T22:45:03.192 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3904 0.73 MB/sec execute 233 sec latency 1495.647 ms 2024-01-21T22:45:04.192 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3906 0.72 MB/sec execute 234 sec latency 1521.459 ms 2024-01-21T22:45:05.192 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3910 0.72 MB/sec execute 235 sec latency 1749.226 ms 2024-01-21T22:45:06.192 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3915 0.72 MB/sec execute 236 sec latency 1394.528 ms 2024-01-21T22:45:07.193 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3920 0.72 MB/sec execute 237 sec latency 1469.511 ms 2024-01-21T22:45:08.193 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3924 0.71 MB/sec execute 238 sec latency 1088.730 ms 2024-01-21T22:45:09.193 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3927 0.71 MB/sec execute 239 sec latency 1634.965 ms 2024-01-21T22:45:10.193 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3929 0.71 MB/sec execute 240 sec latency 1659.040 ms 2024-01-21T22:45:10.758 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:45:10.766 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:45:11.193 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3933 0.70 MB/sec execute 241 sec latency 1528.009 ms 2024-01-21T22:45:12.193 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3937 0.70 MB/sec execute 242 sec latency 1593.669 ms 2024-01-21T22:45:13.193 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3941 0.70 MB/sec execute 243 sec latency 966.488 ms 2024-01-21T22:45:14.194 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3945 0.70 MB/sec execute 244 sec latency 1310.442 ms 2024-01-21T22:45:15.194 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3949 0.69 MB/sec execute 245 sec latency 1027.145 ms 2024-01-21T22:45:16.194 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3954 0.69 MB/sec execute 246 sec latency 1296.096 ms 2024-01-21T22:45:17.195 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3957 0.69 MB/sec execute 247 sec latency 1397.551 ms 2024-01-21T22:45:18.194 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3965 0.69 MB/sec execute 248 sec latency 1453.180 ms 2024-01-21T22:45:19.194 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3967 0.68 MB/sec execute 249 sec latency 1540.066 ms 2024-01-21T22:45:20.194 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3971 0.68 MB/sec execute 250 sec latency 1564.728 ms 2024-01-21T22:45:21.194 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3975 0.68 MB/sec execute 251 sec latency 1236.183 ms 2024-01-21T22:45:22.195 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3980 0.68 MB/sec execute 252 sec latency 1278.484 ms 2024-01-21T22:45:23.195 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3985 0.67 MB/sec execute 253 sec latency 1089.165 ms 2024-01-21T22:45:24.195 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3989 0.67 MB/sec execute 254 sec latency 1211.367 ms 2024-01-21T22:45:25.195 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3995 0.67 MB/sec execute 255 sec latency 926.078 ms 2024-01-21T22:45:26.195 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3998 0.67 MB/sec execute 256 sec latency 1427.679 ms 2024-01-21T22:45:27.195 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4001 0.66 MB/sec execute 257 sec latency 1422.800 ms 2024-01-21T22:45:28.195 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4012 0.66 MB/sec execute 258 sec latency 1631.468 ms 2024-01-21T22:45:29.195 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4022 0.66 MB/sec execute 259 sec latency 1408.318 ms 2024-01-21T22:45:30.195 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4030 0.66 MB/sec execute 260 sec latency 1629.392 ms 2024-01-21T22:45:31.196 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4035 0.65 MB/sec execute 261 sec latency 1271.560 ms 2024-01-21T22:45:32.196 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4046 0.65 MB/sec execute 262 sec latency 1077.024 ms 2024-01-21T22:45:33.196 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4050 0.65 MB/sec execute 263 sec latency 1232.716 ms 2024-01-21T22:45:34.196 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4053 0.65 MB/sec execute 264 sec latency 1383.670 ms 2024-01-21T22:45:35.196 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4057 0.64 MB/sec execute 265 sec latency 1280.763 ms 2024-01-21T22:45:36.196 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4060 0.64 MB/sec execute 266 sec latency 1242.842 ms 2024-01-21T22:45:37.197 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4063 0.64 MB/sec execute 267 sec latency 1324.392 ms 2024-01-21T22:45:38.197 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4067 0.64 MB/sec execute 268 sec latency 1453.474 ms 2024-01-21T22:45:39.197 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4070 0.63 MB/sec execute 269 sec latency 1470.423 ms 2024-01-21T22:45:40.197 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4073 0.63 MB/sec execute 270 sec latency 1582.919 ms 2024-01-21T22:45:40.897 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:45:40.905 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:45:41.197 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4076 0.63 MB/sec execute 271 sec latency 1801.031 ms 2024-01-21T22:45:42.197 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4083 0.63 MB/sec execute 272 sec latency 2017.550 ms 2024-01-21T22:45:43.197 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4089 0.63 MB/sec execute 273 sec latency 2031.833 ms 2024-01-21T22:45:44.197 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4093 0.62 MB/sec execute 274 sec latency 1320.372 ms 2024-01-21T22:45:45.198 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4097 0.62 MB/sec execute 275 sec latency 1734.253 ms 2024-01-21T22:45:46.198 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4102 0.62 MB/sec execute 276 sec latency 2102.241 ms 2024-01-21T22:45:47.198 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4105 0.62 MB/sec execute 277 sec latency 2112.333 ms 2024-01-21T22:45:48.198 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4107 0.61 MB/sec execute 278 sec latency 2113.379 ms 2024-01-21T22:45:49.198 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4115 0.61 MB/sec execute 279 sec latency 2171.331 ms 2024-01-21T22:45:50.198 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4118 0.61 MB/sec execute 280 sec latency 1342.146 ms 2024-01-21T22:45:51.199 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4121 0.61 MB/sec execute 281 sec latency 1427.842 ms 2024-01-21T22:45:52.199 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4126 0.61 MB/sec execute 282 sec latency 1500.057 ms 2024-01-21T22:45:53.199 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4133 0.60 MB/sec execute 283 sec latency 1515.251 ms 2024-01-21T22:45:54.199 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4141 0.60 MB/sec execute 284 sec latency 1427.253 ms 2024-01-21T22:45:55.199 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4148 0.60 MB/sec execute 285 sec latency 1478.620 ms 2024-01-21T22:45:56.199 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4156 0.60 MB/sec execute 286 sec latency 2403.771 ms 2024-01-21T22:45:57.199 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4166 0.60 MB/sec execute 287 sec latency 2568.298 ms 2024-01-21T22:45:58.199 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4176 0.60 MB/sec execute 288 sec latency 1141.279 ms 2024-01-21T22:45:59.200 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4178 0.59 MB/sec execute 289 sec latency 1639.654 ms 2024-01-21T22:46:00.200 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4184 0.59 MB/sec execute 290 sec latency 1905.784 ms 2024-01-21T22:46:01.200 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4200 0.59 MB/sec execute 291 sec latency 975.265 ms 2024-01-21T22:46:02.203 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4204 0.59 MB/sec execute 292 sec latency 1884.429 ms 2024-01-21T22:46:03.200 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4209 0.59 MB/sec execute 293 sec latency 2411.180 ms 2024-01-21T22:46:04.200 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4214 0.59 MB/sec execute 294 sec latency 1845.225 ms 2024-01-21T22:46:05.201 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4218 0.58 MB/sec execute 295 sec latency 1484.721 ms 2024-01-21T22:46:06.201 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4225 0.58 MB/sec execute 296 sec latency 1518.344 ms 2024-01-21T22:46:07.201 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4243 0.58 MB/sec execute 297 sec latency 1242.579 ms 2024-01-21T22:46:08.201 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4248 0.58 MB/sec execute 298 sec latency 1609.119 ms 2024-01-21T22:46:09.201 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4259 0.58 MB/sec execute 299 sec latency 1627.985 ms 2024-01-21T22:46:10.201 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4262 0.58 MB/sec execute 300 sec latency 1808.664 ms 2024-01-21T22:46:11.036 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:46:11.045 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:46:11.201 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4264 0.58 MB/sec execute 301 sec latency 2096.246 ms 2024-01-21T22:46:12.202 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4271 0.57 MB/sec execute 302 sec latency 1489.292 ms 2024-01-21T22:46:13.202 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4289 0.57 MB/sec execute 303 sec latency 1474.179 ms 2024-01-21T22:46:14.202 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4296 0.57 MB/sec execute 304 sec latency 1429.427 ms 2024-01-21T22:46:15.202 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4300 0.57 MB/sec execute 305 sec latency 1955.820 ms 2024-01-21T22:46:16.202 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4321 0.57 MB/sec execute 306 sec latency 1453.949 ms 2024-01-21T22:46:17.202 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4325 0.57 MB/sec execute 307 sec latency 1410.310 ms 2024-01-21T22:46:18.202 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4340 0.57 MB/sec execute 308 sec latency 1201.166 ms 2024-01-21T22:46:19.202 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4345 0.57 MB/sec execute 309 sec latency 1176.521 ms 2024-01-21T22:46:20.203 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4347 0.56 MB/sec execute 310 sec latency 1593.051 ms 2024-01-21T22:46:21.203 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4347 0.56 MB/sec execute 311 sec latency 2593.185 ms 2024-01-21T22:46:22.203 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4352 0.56 MB/sec execute 312 sec latency 3530.914 ms 2024-01-21T22:46:23.203 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4367 0.56 MB/sec execute 313 sec latency 3678.777 ms 2024-01-21T22:46:24.203 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4373 0.56 MB/sec execute 314 sec latency 1277.573 ms 2024-01-21T22:46:25.203 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4381 0.56 MB/sec execute 315 sec latency 1785.905 ms 2024-01-21T22:46:26.203 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4386 0.56 MB/sec execute 316 sec latency 1801.868 ms 2024-01-21T22:46:27.204 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4393 0.56 MB/sec execute 317 sec latency 1674.688 ms 2024-01-21T22:46:28.204 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4400 0.55 MB/sec execute 318 sec latency 1746.409 ms 2024-01-21T22:46:29.204 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4412 0.55 MB/sec execute 319 sec latency 1828.986 ms 2024-01-21T22:46:30.204 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4418 0.55 MB/sec execute 320 sec latency 1725.104 ms 2024-01-21T22:46:31.204 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4428 0.55 MB/sec execute 321 sec latency 1774.822 ms 2024-01-21T22:46:32.204 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4442 0.55 MB/sec execute 322 sec latency 1645.885 ms 2024-01-21T22:46:33.204 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4451 0.55 MB/sec execute 323 sec latency 1667.872 ms 2024-01-21T22:46:34.205 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4455 0.55 MB/sec execute 324 sec latency 1286.599 ms 2024-01-21T22:46:35.205 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4465 0.55 MB/sec execute 325 sec latency 1441.980 ms 2024-01-21T22:46:36.205 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4478 0.55 MB/sec execute 326 sec latency 1118.515 ms 2024-01-21T22:46:37.205 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4486 0.55 MB/sec execute 327 sec latency 1199.306 ms 2024-01-21T22:46:38.205 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4503 0.55 MB/sec execute 328 sec latency 1091.461 ms 2024-01-21T22:46:39.205 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4520 0.56 MB/sec execute 329 sec latency 689.502 ms 2024-01-21T22:46:40.205 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4529 0.56 MB/sec execute 330 sec latency 1040.428 ms 2024-01-21T22:46:41.205 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4538 0.56 MB/sec execute 331 sec latency 1174.120 ms 2024-01-21T22:46:41.342 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:46:41.349 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:46:42.206 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4546 0.56 MB/sec execute 332 sec latency 1456.152 ms 2024-01-21T22:46:43.206 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4553 0.56 MB/sec execute 333 sec latency 1225.438 ms 2024-01-21T22:46:44.206 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4557 0.55 MB/sec execute 334 sec latency 1648.044 ms 2024-01-21T22:46:45.206 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4561 0.55 MB/sec execute 335 sec latency 2134.853 ms 2024-01-21T22:46:46.206 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4569 0.55 MB/sec execute 336 sec latency 1593.771 ms 2024-01-21T22:46:47.206 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4585 0.55 MB/sec execute 337 sec latency 1937.155 ms 2024-01-21T22:46:48.206 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4596 0.55 MB/sec execute 338 sec latency 1959.059 ms 2024-01-21T22:46:49.207 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4615 0.55 MB/sec execute 339 sec latency 1060.037 ms 2024-01-21T22:46:50.207 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4624 0.55 MB/sec execute 340 sec latency 1219.435 ms 2024-01-21T22:46:51.207 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4636 0.55 MB/sec execute 341 sec latency 1410.782 ms 2024-01-21T22:46:52.207 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4650 0.55 MB/sec execute 342 sec latency 1040.821 ms 2024-01-21T22:46:53.207 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4661 0.55 MB/sec execute 343 sec latency 946.274 ms 2024-01-21T22:46:54.207 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4669 0.55 MB/sec execute 344 sec latency 966.895 ms 2024-01-21T22:46:55.207 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4674 0.55 MB/sec execute 345 sec latency 1217.345 ms 2024-01-21T22:46:56.208 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4688 0.55 MB/sec execute 346 sec latency 1186.622 ms 2024-01-21T22:46:57.208 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4702 0.55 MB/sec execute 347 sec latency 979.580 ms 2024-01-21T22:46:58.208 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4715 0.55 MB/sec execute 348 sec latency 1138.550 ms 2024-01-21T22:46:59.208 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4724 0.54 MB/sec execute 349 sec latency 1180.245 ms 2024-01-21T22:47:00.208 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4735 0.54 MB/sec execute 350 sec latency 1085.473 ms 2024-01-21T22:47:01.208 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4735 0.54 MB/sec execute 351 sec latency 1975.177 ms 2024-01-21T22:47:02.208 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4739 0.54 MB/sec execute 352 sec latency 2618.732 ms 2024-01-21T22:47:03.209 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4741 0.54 MB/sec execute 353 sec latency 3261.698 ms 2024-01-21T22:47:04.209 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4754 0.54 MB/sec execute 354 sec latency 3313.580 ms 2024-01-21T22:47:05.209 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4764 0.54 MB/sec execute 355 sec latency 1259.587 ms 2024-01-21T22:47:06.209 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4783 0.55 MB/sec execute 356 sec latency 1356.775 ms 2024-01-21T22:47:07.209 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4809 0.56 MB/sec execute 357 sec latency 1108.359 ms 2024-01-21T22:47:08.209 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4820 0.57 MB/sec execute 358 sec latency 1207.678 ms 2024-01-21T22:47:09.209 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4837 0.57 MB/sec execute 359 sec latency 1875.431 ms 2024-01-21T22:47:10.210 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4852 0.58 MB/sec execute 360 sec latency 1590.876 ms 2024-01-21T22:47:11.210 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4856 0.58 MB/sec execute 361 sec latency 2156.107 ms 2024-01-21T22:47:11.462 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:47:11.470 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:47:12.210 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4863 0.57 MB/sec execute 362 sec latency 2313.667 ms 2024-01-21T22:47:13.210 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4874 0.57 MB/sec execute 363 sec latency 1409.556 ms 2024-01-21T22:47:14.210 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4886 0.57 MB/sec execute 364 sec latency 1025.674 ms 2024-01-21T22:47:15.210 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4896 0.57 MB/sec execute 365 sec latency 1002.863 ms 2024-01-21T22:47:16.210 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4908 0.57 MB/sec execute 366 sec latency 1138.014 ms 2024-01-21T22:47:17.210 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4914 0.57 MB/sec execute 367 sec latency 1524.423 ms 2024-01-21T22:47:18.211 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4923 0.57 MB/sec execute 368 sec latency 1819.385 ms 2024-01-21T22:47:19.211 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4940 0.57 MB/sec execute 369 sec latency 868.025 ms 2024-01-21T22:47:20.211 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4956 0.57 MB/sec execute 370 sec latency 1195.982 ms 2024-01-21T22:47:21.211 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4963 0.57 MB/sec execute 371 sec latency 1325.219 ms 2024-01-21T22:47:22.211 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4973 0.57 MB/sec execute 372 sec latency 1448.699 ms 2024-01-21T22:47:23.211 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4983 0.58 MB/sec execute 373 sec latency 1721.558 ms 2024-01-21T22:47:24.211 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4994 0.58 MB/sec execute 374 sec latency 1711.379 ms 2024-01-21T22:47:25.211 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5002 0.58 MB/sec execute 375 sec latency 1414.674 ms 2024-01-21T22:47:26.212 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5022 0.57 MB/sec execute 376 sec latency 1129.497 ms 2024-01-21T22:47:27.212 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5030 0.57 MB/sec execute 377 sec latency 1013.367 ms 2024-01-21T22:47:28.212 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5059 0.58 MB/sec execute 378 sec latency 1186.822 ms 2024-01-21T22:47:29.212 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5067 0.58 MB/sec execute 379 sec latency 1879.378 ms 2024-01-21T22:47:30.212 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5067 0.58 MB/sec execute 380 sec latency 2163.314 ms 2024-01-21T22:47:31.212 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5077 0.58 MB/sec execute 381 sec latency 2717.810 ms 2024-01-21T22:47:32.212 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5090 0.57 MB/sec execute 382 sec latency 2350.608 ms 2024-01-21T22:47:33.213 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5106 0.57 MB/sec execute 383 sec latency 1649.639 ms 2024-01-21T22:47:34.213 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5115 0.57 MB/sec execute 384 sec latency 1343.277 ms 2024-01-21T22:47:35.213 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5123 0.57 MB/sec execute 385 sec latency 1359.419 ms 2024-01-21T22:47:36.213 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5134 0.57 MB/sec execute 386 sec latency 1331.153 ms 2024-01-21T22:47:37.213 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5147 0.57 MB/sec execute 387 sec latency 1103.670 ms 2024-01-21T22:47:38.213 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5151 0.57 MB/sec execute 388 sec latency 1378.036 ms 2024-01-21T22:47:39.213 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5158 0.56 MB/sec execute 389 sec latency 1582.584 ms 2024-01-21T22:47:40.214 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5167 0.56 MB/sec execute 390 sec latency 1471.615 ms 2024-01-21T22:47:41.214 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5188 0.56 MB/sec execute 391 sec latency 1041.846 ms 2024-01-21T22:47:41.609 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:47:41.618 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:47:42.214 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5206 0.56 MB/sec execute 392 sec latency 1202.455 ms 2024-01-21T22:47:43.214 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5228 0.57 MB/sec execute 393 sec latency 1596.146 ms 2024-01-21T22:47:44.214 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5240 0.57 MB/sec execute 394 sec latency 2183.524 ms 2024-01-21T22:47:45.214 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5254 0.57 MB/sec execute 395 sec latency 1734.335 ms 2024-01-21T22:47:46.214 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5271 0.56 MB/sec execute 396 sec latency 1597.688 ms 2024-01-21T22:47:47.215 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5283 0.56 MB/sec execute 397 sec latency 1467.065 ms 2024-01-21T22:47:48.215 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5301 0.56 MB/sec execute 398 sec latency 1446.969 ms 2024-01-21T22:47:49.215 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5320 0.56 MB/sec execute 399 sec latency 1068.588 ms 2024-01-21T22:47:50.215 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5337 0.56 MB/sec execute 400 sec latency 1012.298 ms 2024-01-21T22:47:51.215 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5345 0.56 MB/sec execute 401 sec latency 1548.871 ms 2024-01-21T22:47:52.215 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5368 0.56 MB/sec execute 402 sec latency 1798.725 ms 2024-01-21T22:47:53.215 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5384 0.56 MB/sec execute 403 sec latency 1579.938 ms 2024-01-21T22:47:54.216 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5397 0.56 MB/sec execute 404 sec latency 1527.419 ms 2024-01-21T22:47:55.216 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5455 0.56 MB/sec execute 405 sec latency 1157.204 ms 2024-01-21T22:47:56.216 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5480 0.56 MB/sec execute 406 sec latency 1555.283 ms 2024-01-21T22:47:57.216 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5500 0.55 MB/sec execute 407 sec latency 1600.802 ms 2024-01-21T22:47:58.216 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5540 0.55 MB/sec execute 408 sec latency 1394.531 ms 2024-01-21T22:47:59.216 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5565 0.55 MB/sec execute 409 sec latency 955.522 ms 2024-01-21T22:48:00.216 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5598 0.55 MB/sec execute 410 sec latency 941.414 ms 2024-01-21T22:48:01.216 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5636 0.56 MB/sec execute 411 sec latency 1006.806 ms 2024-01-21T22:48:01.587 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:23:45.780 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:48:01.603 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:23:45.861 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:48:01.604 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:23:45.862 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:48:01.606 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:23:45.864 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:48:01.606 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:23:45.799 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:48:01.607 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:23:45.865 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:48:01.746 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:23:46.003 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:48:01.759 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:23:46.017 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:48:01.821 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:23:46.015 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:48:02.217 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5646 0.56 MB/sec execute 412 sec latency 1355.837 ms 2024-01-21T22:48:03.217 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5659 0.56 MB/sec execute 413 sec latency 1755.594 ms 2024-01-21T22:48:04.217 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5682 0.56 MB/sec execute 414 sec latency 1316.587 ms 2024-01-21T22:48:05.217 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5715 0.56 MB/sec execute 415 sec latency 896.773 ms 2024-01-21T22:48:06.217 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5730 0.56 MB/sec execute 416 sec latency 1087.832 ms 2024-01-21T22:48:07.217 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5745 0.56 MB/sec execute 417 sec latency 1448.544 ms 2024-01-21T22:48:08.217 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5774 0.56 MB/sec execute 418 sec latency 883.419 ms 2024-01-21T22:48:09.218 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5786 0.56 MB/sec execute 419 sec latency 1268.039 ms 2024-01-21T22:48:10.218 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5804 0.56 MB/sec execute 420 sec latency 1561.573 ms 2024-01-21T22:48:11.218 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5811 0.56 MB/sec execute 421 sec latency 1706.587 ms 2024-01-21T22:48:11.814 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:48:11.822 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:48:12.218 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5811 0.56 MB/sec execute 422 sec latency 2706.727 ms 2024-01-21T22:48:13.218 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5811 0.56 MB/sec execute 423 sec latency 3706.871 ms 2024-01-21T22:48:14.218 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5811 0.55 MB/sec execute 424 sec latency 4707.023 ms 2024-01-21T22:48:15.218 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5816 0.55 MB/sec execute 425 sec latency 5601.023 ms 2024-01-21T22:48:16.219 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5830 0.56 MB/sec execute 426 sec latency 5981.719 ms 2024-01-21T22:48:17.219 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5840 0.56 MB/sec execute 427 sec latency 1782.913 ms 2024-01-21T22:48:18.219 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5856 0.55 MB/sec execute 428 sec latency 1961.649 ms 2024-01-21T22:48:19.219 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5867 0.55 MB/sec execute 429 sec latency 2162.292 ms 2024-01-21T22:48:20.219 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5880 0.55 MB/sec execute 430 sec latency 1054.545 ms 2024-01-21T22:48:21.219 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5895 0.55 MB/sec execute 431 sec latency 1009.846 ms 2024-01-21T22:48:22.219 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5907 0.55 MB/sec execute 432 sec latency 1357.052 ms 2024-01-21T22:48:23.219 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5914 0.55 MB/sec execute 433 sec latency 1653.300 ms 2024-01-21T22:48:24.220 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5924 0.55 MB/sec execute 434 sec latency 1778.155 ms 2024-01-21T22:48:25.220 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5931 0.55 MB/sec execute 435 sec latency 1313.095 ms 2024-01-21T22:48:26.220 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5943 0.55 MB/sec execute 436 sec latency 1351.105 ms 2024-01-21T22:48:27.220 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5951 0.55 MB/sec execute 437 sec latency 1128.828 ms 2024-01-21T22:48:28.220 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5966 0.55 MB/sec execute 438 sec latency 1282.716 ms 2024-01-21T22:48:29.220 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5980 0.55 MB/sec execute 439 sec latency 909.607 ms 2024-01-21T22:48:30.220 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5989 0.55 MB/sec execute 440 sec latency 1212.902 ms 2024-01-21T22:48:31.221 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5998 0.55 MB/sec execute 441 sec latency 1400.794 ms 2024-01-21T22:48:32.221 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6011 0.55 MB/sec execute 442 sec latency 1043.210 ms 2024-01-21T22:48:33.221 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6022 0.55 MB/sec execute 443 sec latency 1242.678 ms 2024-01-21T22:48:34.221 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6039 0.55 MB/sec execute 444 sec latency 1051.768 ms 2024-01-21T22:48:35.221 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6045 0.55 MB/sec execute 445 sec latency 895.803 ms 2024-01-21T22:48:36.221 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6060 0.55 MB/sec execute 446 sec latency 1016.027 ms 2024-01-21T22:48:37.221 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6063 0.55 MB/sec execute 447 sec latency 1334.394 ms 2024-01-21T22:48:38.221 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6065 0.55 MB/sec execute 448 sec latency 2334.525 ms 2024-01-21T22:48:39.222 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6067 0.55 MB/sec execute 449 sec latency 3043.464 ms 2024-01-21T22:48:40.221 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6081 0.55 MB/sec execute 450 sec latency 3755.549 ms 2024-01-21T22:48:41.222 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6088 0.55 MB/sec execute 451 sec latency 1655.868 ms 2024-01-21T22:48:41.953 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:48:41.961 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:48:42.222 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6093 0.55 MB/sec execute 452 sec latency 1283.776 ms 2024-01-21T22:48:43.222 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6099 0.55 MB/sec execute 453 sec latency 1021.522 ms 2024-01-21T22:48:44.222 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6112 0.55 MB/sec execute 454 sec latency 1040.597 ms 2024-01-21T22:48:45.222 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6132 0.56 MB/sec execute 455 sec latency 1139.946 ms 2024-01-21T22:48:46.222 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6158 0.57 MB/sec execute 456 sec latency 1482.024 ms 2024-01-21T22:48:46.912 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:24:31.105 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:48:46.920 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:24:31.113 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:48:46.955 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:24:31.213 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:48:46.960 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:24:31.218 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:48:46.962 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:24:31.155 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:48:46.967 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:24:31.225 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:48:46.973 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:24:31.231 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:48:46.992 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:24:31.250 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:48:47.020 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:24:31.278 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:48:47.223 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6167 0.57 MB/sec execute 457 sec latency 1563.885 ms 2024-01-21T22:48:48.223 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6178 0.57 MB/sec execute 458 sec latency 1874.522 ms 2024-01-21T22:48:49.223 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6191 0.57 MB/sec execute 459 sec latency 1434.683 ms 2024-01-21T22:48:50.223 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6197 0.57 MB/sec execute 460 sec latency 1250.947 ms 2024-01-21T22:48:51.223 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6205 0.57 MB/sec execute 461 sec latency 1022.778 ms 2024-01-21T22:48:52.223 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6221 0.57 MB/sec execute 462 sec latency 1066.163 ms 2024-01-21T22:48:53.224 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6241 0.57 MB/sec execute 463 sec latency 1191.365 ms 2024-01-21T22:48:54.223 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6267 0.58 MB/sec execute 464 sec latency 1293.345 ms 2024-01-21T22:48:55.223 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6296 0.59 MB/sec execute 465 sec latency 1414.816 ms 2024-01-21T22:48:56.224 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6314 0.59 MB/sec execute 466 sec latency 1319.222 ms 2024-01-21T22:48:57.224 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6325 0.59 MB/sec execute 467 sec latency 1199.319 ms 2024-01-21T22:48:58.224 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6327 0.59 MB/sec execute 468 sec latency 1589.558 ms 2024-01-21T22:48:59.224 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6336 0.60 MB/sec execute 469 sec latency 1694.112 ms 2024-01-21T22:49:00.224 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6347 0.59 MB/sec execute 470 sec latency 1409.608 ms 2024-01-21T22:49:01.225 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6360 0.60 MB/sec execute 471 sec latency 1423.102 ms 2024-01-21T22:49:02.224 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6380 0.60 MB/sec execute 472 sec latency 1143.940 ms 2024-01-21T22:49:03.225 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6391 0.60 MB/sec execute 473 sec latency 1155.613 ms 2024-01-21T22:49:04.225 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6408 0.60 MB/sec execute 474 sec latency 1400.042 ms 2024-01-21T22:49:05.225 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6426 0.60 MB/sec execute 475 sec latency 657.457 ms 2024-01-21T22:49:06.225 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6436 0.60 MB/sec execute 476 sec latency 1322.337 ms 2024-01-21T22:49:07.225 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6447 0.61 MB/sec execute 477 sec latency 1447.929 ms 2024-01-21T22:49:08.225 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6455 0.61 MB/sec execute 478 sec latency 1294.360 ms 2024-01-21T22:49:09.225 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6463 0.60 MB/sec execute 479 sec latency 1154.439 ms 2024-01-21T22:49:10.226 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6474 0.60 MB/sec execute 480 sec latency 1250.056 ms 2024-01-21T22:49:11.226 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6484 0.60 MB/sec execute 481 sec latency 897.179 ms 2024-01-21T22:49:12.134 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:49:12.141 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:49:12.226 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6495 0.60 MB/sec execute 482 sec latency 1422.927 ms 2024-01-21T22:49:13.226 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6527 0.60 MB/sec execute 483 sec latency 1128.309 ms 2024-01-21T22:49:14.226 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6530 0.60 MB/sec execute 484 sec latency 2059.854 ms 2024-01-21T22:49:15.226 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6536 0.60 MB/sec execute 485 sec latency 1162.430 ms 2024-01-21T22:49:16.226 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6571 0.61 MB/sec execute 486 sec latency 1182.851 ms 2024-01-21T22:49:17.227 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6576 0.61 MB/sec execute 487 sec latency 1327.805 ms 2024-01-21T22:49:18.227 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6586 0.61 MB/sec execute 488 sec latency 1574.454 ms 2024-01-21T22:49:19.227 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6594 0.61 MB/sec execute 489 sec latency 1006.161 ms 2024-01-21T22:49:20.227 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6648 0.61 MB/sec execute 490 sec latency 1216.455 ms 2024-01-21T22:49:21.227 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6684 0.61 MB/sec execute 491 sec latency 923.040 ms 2024-01-21T22:49:22.227 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6692 0.62 MB/sec execute 492 sec latency 1191.548 ms 2024-01-21T22:49:23.227 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6711 0.61 MB/sec execute 493 sec latency 1344.428 ms 2024-01-21T22:49:24.227 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6724 0.61 MB/sec execute 494 sec latency 1364.015 ms 2024-01-21T22:49:25.227 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6748 0.62 MB/sec execute 495 sec latency 936.472 ms 2024-01-21T22:49:26.228 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6754 0.61 MB/sec execute 496 sec latency 1038.091 ms 2024-01-21T22:49:27.228 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6758 0.61 MB/sec execute 497 sec latency 1333.998 ms 2024-01-21T22:49:28.228 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6784 0.62 MB/sec execute 498 sec latency 1410.264 ms 2024-01-21T22:49:29.228 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6801 0.62 MB/sec execute 499 sec latency 1043.918 ms 2024-01-21T22:49:30.228 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6823 0.62 MB/sec execute 500 sec latency 934.932 ms 2024-01-21T22:49:31.228 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6840 0.63 MB/sec execute 501 sec latency 1059.649 ms 2024-01-21T22:49:32.229 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6855 0.63 MB/sec execute 502 sec latency 1339.265 ms 2024-01-21T22:49:33.229 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6883 0.64 MB/sec execute 503 sec latency 1882.858 ms 2024-01-21T22:49:34.229 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6884 0.63 MB/sec execute 504 sec latency 1907.589 ms 2024-01-21T22:49:35.229 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6887 0.63 MB/sec execute 505 sec latency 2626.160 ms 2024-01-21T22:49:36.229 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6898 0.63 MB/sec execute 506 sec latency 2635.852 ms 2024-01-21T22:49:37.229 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6903 0.63 MB/sec execute 507 sec latency 1417.971 ms 2024-01-21T22:49:38.229 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6908 0.63 MB/sec execute 508 sec latency 1462.185 ms 2024-01-21T22:49:39.230 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6918 0.63 MB/sec execute 509 sec latency 1404.255 ms 2024-01-21T22:49:40.230 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6923 0.63 MB/sec execute 510 sec latency 1569.535 ms 2024-01-21T22:49:41.230 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6939 0.63 MB/sec execute 511 sec latency 1605.339 ms 2024-01-21T22:49:42.230 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6950 0.64 MB/sec execute 512 sec latency 1480.281 ms 2024-01-21T22:49:42.255 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:49:42.263 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:49:43.230 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6965 0.64 MB/sec execute 513 sec latency 1616.782 ms 2024-01-21T22:49:44.230 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6978 0.65 MB/sec execute 514 sec latency 1846.191 ms 2024-01-21T22:49:45.230 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6986 0.65 MB/sec execute 515 sec latency 2218.407 ms 2024-01-21T22:49:46.231 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7011 0.65 MB/sec execute 516 sec latency 987.772 ms 2024-01-21T22:49:47.231 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7032 0.65 MB/sec execute 517 sec latency 1054.716 ms 2024-01-21T22:49:48.231 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7036 0.65 MB/sec execute 518 sec latency 1150.066 ms 2024-01-21T22:49:49.231 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7045 0.65 MB/sec execute 519 sec latency 1171.474 ms 2024-01-21T22:49:50.231 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7048 0.65 MB/sec execute 520 sec latency 1744.732 ms 2024-01-21T22:49:51.231 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7055 0.64 MB/sec execute 521 sec latency 2744.872 ms 2024-01-21T22:49:52.231 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7066 0.65 MB/sec execute 522 sec latency 3302.000 ms 2024-01-21T22:49:53.231 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7071 0.65 MB/sec execute 523 sec latency 1680.695 ms 2024-01-21T22:49:54.232 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7081 0.65 MB/sec execute 524 sec latency 1549.596 ms 2024-01-21T22:49:55.232 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7092 0.65 MB/sec execute 525 sec latency 1181.538 ms 2024-01-21T22:49:56.232 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7107 0.65 MB/sec execute 526 sec latency 1243.180 ms 2024-01-21T22:49:57.232 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7119 0.65 MB/sec execute 527 sec latency 1123.286 ms 2024-01-21T22:49:58.232 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7127 0.65 MB/sec execute 528 sec latency 1413.775 ms 2024-01-21T22:49:59.232 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7144 0.65 MB/sec execute 529 sec latency 1495.635 ms 2024-01-21T22:50:00.232 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7150 0.65 MB/sec execute 530 sec latency 1349.644 ms 2024-01-21T22:50:01.233 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7159 0.65 MB/sec execute 531 sec latency 1325.987 ms 2024-01-21T22:50:02.233 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7166 0.65 MB/sec execute 532 sec latency 1075.428 ms 2024-01-21T22:50:03.233 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7169 0.65 MB/sec execute 533 sec latency 1257.392 ms 2024-01-21T22:50:04.233 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7181 0.65 MB/sec execute 534 sec latency 1696.933 ms 2024-01-21T22:50:05.233 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7188 0.64 MB/sec execute 535 sec latency 1217.068 ms 2024-01-21T22:50:06.233 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7194 0.65 MB/sec execute 536 sec latency 1154.756 ms 2024-01-21T22:50:07.233 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7197 0.64 MB/sec execute 537 sec latency 2154.897 ms 2024-01-21T22:50:08.234 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7205 0.64 MB/sec execute 538 sec latency 2780.984 ms 2024-01-21T22:50:09.234 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7210 0.64 MB/sec execute 539 sec latency 1603.395 ms 2024-01-21T22:50:10.234 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7218 0.64 MB/sec execute 540 sec latency 1608.888 ms 2024-01-21T22:50:11.234 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7233 0.65 MB/sec execute 541 sec latency 1034.819 ms 2024-01-21T22:50:12.234 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7245 0.65 MB/sec execute 542 sec latency 1219.854 ms 2024-01-21T22:50:12.403 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:50:12.411 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:50:12.542 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:25:56.736 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:50:12.644 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:25:56.837 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:50:12.751 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:25:56.944 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:50:12.770 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:25:57.029 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:50:12.775 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:25:57.034 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:50:12.776 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:25:57.035 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:50:12.781 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:25:57.039 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:50:12.810 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:25:57.068 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:50:12.820 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:25:57.078 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:50:13.234 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7250 0.65 MB/sec execute 543 sec latency 1334.517 ms 2024-01-21T22:50:14.234 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7261 0.65 MB/sec execute 544 sec latency 1653.686 ms 2024-01-21T22:50:15.235 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7267 0.65 MB/sec execute 545 sec latency 1624.150 ms 2024-01-21T22:50:16.235 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7269 0.65 MB/sec execute 546 sec latency 1250.289 ms 2024-01-21T22:50:17.235 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7271 0.65 MB/sec execute 547 sec latency 1941.190 ms 2024-01-21T22:50:18.235 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7273 0.65 MB/sec execute 548 sec latency 2781.596 ms 2024-01-21T22:50:19.236 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7280 0.65 MB/sec execute 549 sec latency 2838.839 ms 2024-01-21T22:50:20.236 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7285 0.65 MB/sec execute 550 sec latency 1311.421 ms 2024-01-21T22:50:21.236 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7288 0.64 MB/sec execute 551 sec latency 1300.316 ms 2024-01-21T22:50:22.236 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7294 0.64 MB/sec execute 552 sec latency 1386.262 ms 2024-01-21T22:50:23.236 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7305 0.64 MB/sec execute 553 sec latency 1263.930 ms 2024-01-21T22:50:24.236 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7310 0.64 MB/sec execute 554 sec latency 1693.743 ms 2024-01-21T22:50:25.236 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7312 0.64 MB/sec execute 555 sec latency 1326.973 ms 2024-01-21T22:50:26.236 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7316 0.64 MB/sec execute 556 sec latency 1554.367 ms 2024-01-21T22:50:27.237 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7320 0.64 MB/sec execute 557 sec latency 1190.731 ms 2024-01-21T22:50:28.237 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7325 0.64 MB/sec execute 558 sec latency 1427.633 ms 2024-01-21T22:50:29.237 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7329 0.64 MB/sec execute 559 sec latency 1381.361 ms 2024-01-21T22:50:30.237 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7332 0.64 MB/sec execute 560 sec latency 1061.621 ms 2024-01-21T22:50:31.237 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7337 0.64 MB/sec execute 561 sec latency 1146.013 ms 2024-01-21T22:50:32.237 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7342 0.63 MB/sec execute 562 sec latency 1249.783 ms 2024-01-21T22:50:33.237 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7345 0.63 MB/sec execute 563 sec latency 1257.590 ms 2024-01-21T22:50:34.237 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7347 0.63 MB/sec execute 564 sec latency 1651.256 ms 2024-01-21T22:50:35.238 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7350 0.63 MB/sec execute 565 sec latency 1653.365 ms 2024-01-21T22:50:36.238 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7356 0.63 MB/sec execute 566 sec latency 1193.525 ms 2024-01-21T22:50:37.238 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7364 0.63 MB/sec execute 567 sec latency 899.058 ms 2024-01-21T22:50:38.238 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7371 0.63 MB/sec execute 568 sec latency 994.470 ms 2024-01-21T22:50:39.238 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7375 0.63 MB/sec execute 569 sec latency 1283.592 ms 2024-01-21T22:50:40.238 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7379 0.63 MB/sec execute 570 sec latency 1172.464 ms 2024-01-21T22:50:41.238 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7383 0.63 MB/sec execute 571 sec latency 1452.476 ms 2024-01-21T22:50:42.238 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7388 0.63 MB/sec execute 572 sec latency 1563.713 ms 2024-01-21T22:50:42.568 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:50:42.576 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:50:43.239 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7395 0.63 MB/sec execute 573 sec latency 994.220 ms 2024-01-21T22:50:44.239 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7399 0.63 MB/sec execute 574 sec latency 1122.594 ms 2024-01-21T22:50:45.239 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7403 0.62 MB/sec execute 575 sec latency 1196.029 ms 2024-01-21T22:50:46.239 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7408 0.62 MB/sec execute 576 sec latency 1056.712 ms 2024-01-21T22:50:47.239 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7412 0.62 MB/sec execute 577 sec latency 1303.066 ms 2024-01-21T22:50:48.239 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7418 0.62 MB/sec execute 578 sec latency 1134.781 ms 2024-01-21T22:50:49.239 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7421 0.62 MB/sec execute 579 sec latency 1440.503 ms 2024-01-21T22:50:50.240 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7426 0.62 MB/sec execute 580 sec latency 1652.087 ms 2024-01-21T22:50:51.240 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7428 0.62 MB/sec execute 581 sec latency 1331.220 ms 2024-01-21T22:50:52.240 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7436 0.62 MB/sec execute 582 sec latency 1652.679 ms 2024-01-21T22:50:53.240 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7437 0.62 MB/sec execute 583 sec latency 1339.314 ms 2024-01-21T22:50:54.240 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7444 0.62 MB/sec execute 584 sec latency 1554.012 ms 2024-01-21T22:50:55.240 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7449 0.62 MB/sec execute 585 sec latency 905.692 ms 2024-01-21T22:50:56.240 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7453 0.62 MB/sec execute 586 sec latency 1297.299 ms 2024-01-21T22:50:57.240 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7457 0.62 MB/sec execute 587 sec latency 1372.583 ms 2024-01-21T22:50:58.241 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7461 0.61 MB/sec execute 588 sec latency 1147.596 ms 2024-01-21T22:50:59.241 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7466 0.61 MB/sec execute 589 sec latency 1002.819 ms 2024-01-21T22:51:00.241 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7472 0.61 MB/sec execute 590 sec latency 1308.099 ms 2024-01-21T22:51:01.241 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7473 0.61 MB/sec execute 591 sec latency 1682.404 ms 2024-01-21T22:51:02.241 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7477 0.61 MB/sec execute 592 sec latency 1563.171 ms 2024-01-21T22:51:03.241 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7479 0.61 MB/sec execute 593 sec latency 1413.572 ms 2024-01-21T22:51:04.241 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7483 0.61 MB/sec execute 594 sec latency 1483.356 ms 2024-01-21T22:51:05.242 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7485 0.61 MB/sec execute 595 sec latency 1449.196 ms 2024-01-21T22:51:06.242 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7489 0.61 MB/sec execute 596 sec latency 1548.249 ms 2024-01-21T22:51:07.242 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7493 0.60 MB/sec execute 597 sec latency 1049.097 ms 2024-01-21T22:51:08.242 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7497 0.60 MB/sec execute 598 sec latency 1108.096 ms 2024-01-21T22:51:09.242 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7500 0.60 MB/sec execute 599 sec latency 1463.991 ms 2024-01-21T22:51:10.243 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 600 sec 2024-01-21T22:51:11.242 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 601 sec 2024-01-21T22:51:12.243 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 602 sec 2024-01-21T22:51:12.704 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:51:12.713 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:51:13.243 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 603 sec 2024-01-21T22:51:14.243 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 604 sec 2024-01-21T22:51:15.243 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 605 sec 2024-01-21T22:51:16.243 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 606 sec 2024-01-21T22:51:17.243 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 607 sec 2024-01-21T22:51:18.243 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 608 sec 2024-01-21T22:51:19.243 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 609 sec 2024-01-21T22:51:20.244 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 610 sec 2024-01-21T22:51:21.244 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 611 sec 2024-01-21T22:51:22.244 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 612 sec 2024-01-21T22:51:23.244 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 613 sec 2024-01-21T22:51:24.244 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 614 sec 2024-01-21T22:51:25.244 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 615 sec 2024-01-21T22:51:26.244 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 616 sec 2024-01-21T22:51:27.245 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 617 sec 2024-01-21T22:51:28.030 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:27:12.223 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:51:28.037 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:27:12.295 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:51:28.039 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:27:12.297 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:51:28.045 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:27:12.303 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:51:28.045 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:27:12.238 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:51:28.049 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:27:12.307 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:51:28.082 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:27:12.340 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:51:28.106 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:27:12.364 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:51:28.141 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:27:12.334 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:51:28.244 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 618 sec 2024-01-21T22:51:29.245 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 619 sec 2024-01-21T22:51:30.245 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 620 sec 2024-01-21T22:51:31.245 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 621 sec 2024-01-21T22:51:32.245 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 622 sec 2024-01-21T22:51:33.245 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 623 sec 2024-01-21T22:51:34.247 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 624 sec 2024-01-21T22:51:35.245 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 625 sec 2024-01-21T22:51:36.246 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 626 sec 2024-01-21T22:51:37.246 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 627 sec 2024-01-21T22:51:38.246 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 628 sec 2024-01-21T22:51:39.246 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 629 sec 2024-01-21T22:51:40.246 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 630 sec 2024-01-21T22:51:41.246 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 631 sec 2024-01-21T22:51:42.246 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 632 sec 2024-01-21T22:51:42.842 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:51:42.851 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:51:43.247 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 633 sec 2024-01-21T22:51:44.247 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 634 sec 2024-01-21T22:51:45.247 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 635 sec 2024-01-21T22:51:46.247 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 636 sec 2024-01-21T22:51:47.247 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 637 sec 2024-01-21T22:51:48.247 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 638 sec 2024-01-21T22:51:49.247 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 639 sec 2024-01-21T22:51:50.247 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 640 sec 2024-01-21T22:51:51.247 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 641 sec 2024-01-21T22:51:52.248 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 642 sec 2024-01-21T22:51:53.248 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 643 sec 2024-01-21T22:51:54.248 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 644 sec 2024-01-21T22:51:55.248 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 645 sec 2024-01-21T22:51:56.248 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 646 sec 2024-01-21T22:51:57.248 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 647 sec 2024-01-21T22:51:58.249 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 648 sec 2024-01-21T22:51:59.249 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 649 sec 2024-01-21T22:52:00.249 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 650 sec 2024-01-21T22:52:01.249 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 651 sec 2024-01-21T22:52:02.249 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 652 sec 2024-01-21T22:52:03.249 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 653 sec 2024-01-21T22:52:04.249 INFO:tasks.workunit.client.0.smithi138.stdout: 10 cleanup 654 sec 2024-01-21T22:52:05.249 INFO:tasks.workunit.client.0.smithi138.stdout: 8 cleanup 655 sec 2024-01-21T22:52:06.249 INFO:tasks.workunit.client.0.smithi138.stdout: 8 cleanup 656 sec 2024-01-21T22:52:07.250 INFO:tasks.workunit.client.0.smithi138.stdout: 8 cleanup 657 sec 2024-01-21T22:52:08.250 INFO:tasks.workunit.client.0.smithi138.stdout: 8 cleanup 658 sec 2024-01-21T22:52:09.250 INFO:tasks.workunit.client.0.smithi138.stdout: 8 cleanup 659 sec 2024-01-21T22:52:10.250 INFO:tasks.workunit.client.0.smithi138.stdout: 8 cleanup 660 sec 2024-01-21T22:52:11.250 INFO:tasks.workunit.client.0.smithi138.stdout: 8 cleanup 661 sec 2024-01-21T22:52:12.250 INFO:tasks.workunit.client.0.smithi138.stdout: 7 cleanup 662 sec 2024-01-21T22:52:12.998 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:52:13.007 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:52:13.250 INFO:tasks.workunit.client.0.smithi138.stdout: 7 cleanup 663 sec 2024-01-21T22:52:14.251 INFO:tasks.workunit.client.0.smithi138.stdout: 5 cleanup 664 sec 2024-01-21T22:52:15.251 INFO:tasks.workunit.client.0.smithi138.stdout: 5 cleanup 665 sec 2024-01-21T22:52:16.251 INFO:tasks.workunit.client.0.smithi138.stdout: 5 cleanup 666 sec 2024-01-21T22:52:17.251 INFO:tasks.workunit.client.0.smithi138.stdout: 5 cleanup 667 sec 2024-01-21T22:52:18.251 INFO:tasks.workunit.client.0.smithi138.stdout: 5 cleanup 668 sec 2024-01-21T22:52:19.252 INFO:tasks.workunit.client.0.smithi138.stdout: 4 cleanup 669 sec 2024-01-21T22:52:20.118 INFO:tasks.workunit.client.0.smithi138.stdout: 0 cleanup 670 sec 2024-01-21T22:52:20.119 INFO:tasks.workunit.client.0.smithi138.stdout: 2024-01-21T22:52:20.119 INFO:tasks.workunit.client.0.smithi138.stdout: Operation Count AvgLat MaxLat 2024-01-21T22:52:20.119 INFO:tasks.workunit.client.0.smithi138.stdout: ---------------------------------------- 2024-01-21T22:52:20.119 INFO:tasks.workunit.client.0.smithi138.stdout: NTCreateX 10790 218.388 4889.281 2024-01-21T22:52:20.119 INFO:tasks.workunit.client.0.smithi138.stdout: Close 7992 0.013 0.321 2024-01-21T22:52:20.119 INFO:tasks.workunit.client.0.smithi138.stdout: Rename 453 965.764 5981.698 2024-01-21T22:52:20.119 INFO:tasks.workunit.client.0.smithi138.stdout: Unlink 1980 465.644 5224.753 2024-01-21T22:52:20.119 INFO:tasks.workunit.client.0.smithi138.stdout: Qpathinfo 10071 173.834 5655.088 2024-01-21T22:52:20.119 INFO:tasks.workunit.client.0.smithi138.stdout: Qfileinfo 1814 0.006 0.132 2024-01-21T22:52:20.119 INFO:tasks.workunit.client.0.smithi138.stdout: Qfsinfo 1736 16.946 375.367 2024-01-21T22:52:20.119 INFO:tasks.workunit.client.0.smithi138.stdout: Sfileinfo 756 102.034 4133.215 2024-01-21T22:52:20.119 INFO:tasks.workunit.client.0.smithi138.stdout: Find 3785 20.589 3576.118 2024-01-21T22:52:20.120 INFO:tasks.workunit.client.0.smithi138.stdout: WriteX 5344 14.873 4199.531 2024-01-21T22:52:20.120 INFO:tasks.workunit.client.0.smithi138.stdout: ReadX 17993 4.561 1922.451 2024-01-21T22:52:20.120 INFO:tasks.workunit.client.0.smithi138.stdout: LockX 40 394.568 654.200 2024-01-21T22:52:20.120 INFO:tasks.workunit.client.0.smithi138.stdout: UnlockX 40 360.861 1342.593 2024-01-21T22:52:20.120 INFO:tasks.workunit.client.0.smithi138.stdout: Flush 620 265.545 2222.900 2024-01-21T22:52:20.120 INFO:tasks.workunit.client.0.smithi138.stdout: 2024-01-21T22:52:20.120 INFO:tasks.workunit.client.0.smithi138.stdout:Throughput 0.602796 MB/sec 10 clients 10 procs max_latency=5981.719 ms 2024-01-21T22:52:20.121 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-01-21T22:52:20.122 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-01-21T22:52:35.804 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-01-21T22:52:35.804 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-01-21T22:52:36.772 DEBUG:teuthology.parallel:result is None 2024-01-21T22:52:36.772 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-01-21T22:52:36.968 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-01-21T22:52:36.968 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-01-21T22:52:36.979 INFO:tasks.kclient:Unmounting kernel clients... 2024-01-21T22:52:36.979 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:52:36.979 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-21T22:52:37.034 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-01-21T22:52:37.034 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-01-21T22:52:37.085 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-21T22:52:37.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.293+0000 7f7e43600640 1 Processor -- start 2024-01-21T22:52:37.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.293+0000 7f7e43600640 1 -- start start 2024-01-21T22:52:37.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.294+0000 7f7e43600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e3c0716e0 0x7f7e3c0c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:37.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.294+0000 7f7e43600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7e3c0c7520 0x7f7e3c06ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:37.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.295+0000 7f7e43600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e3c157740 con 0x7f7e3c150a00 2024-01-21T22:52:37.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.295+0000 7f7e43600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e3c0c4cc0 con 0x7f7e3c0c7520 2024-01-21T22:52:37.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.295+0000 7f7e43600640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c06a920 con 0x7f7e3c0716e0 2024-01-21T22:52:37.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.296+0000 7f7e41600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7e3c0c7520 0x7f7e3c06ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:37.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.296+0000 7f7e3be00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e3c0716e0 0x7f7e3c0c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:37.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.296+0000 7f7e3be00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e3c0716e0 0x7f7e3c0c6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:35370/0 (socket says 172.21.15.138:35370) 2024-01-21T22:52:37.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.296+0000 7f7e3be00640 1 -- 172.21.15.138:0/1985264569 learned_addr learned my addr 172.21.15.138:0/1985264569 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:37.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.298+0000 7f7e3be00640 1 -- 172.21.15.138:0/1985264569 >> v2:172.21.15.138:3300/0 conn(0x7f7e3c0c7520 msgr2=0x7f7e3c06ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:37.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.298+0000 7f7e3be00640 1 --2- 172.21.15.138:0/1985264569 >> v2:172.21.15.138:3300/0 conn(0x7f7e3c0c7520 0x7f7e3c06ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:37.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.298+0000 7f7e3be00640 1 -- 172.21.15.138:0/1985264569 >> v1:172.21.15.138:6789/0 conn(0x7f7e3c150a00 legacy=0x7f7e3c0c25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:37.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.298+0000 7f7e3be00640 1 -- 172.21.15.138:0/1985264569 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c156bc0 con 0x7f7e3c0716e0 2024-01-21T22:52:37.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.301+0000 7f7e3be00640 1 --2- 172.21.15.138:0/1985264569 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e3c0716e0 0x7f7e3c0c6fe0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009870 tx=0x7f7e2402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9d7af1d76f7726f server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:37.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.305+0000 7f7e3b400640 1 -- 172.21.15.138:0/1985264569 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24031640 con 0x7f7e3c0716e0 2024-01-21T22:52:37.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.305+0000 7f7e3b400640 1 -- 172.21.15.138:0/1985264569 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e240317a0 con 0x7f7e3c0716e0 2024-01-21T22:52:37.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.305+0000 7f7e3b400640 1 -- 172.21.15.138:0/1985264569 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24031b30 con 0x7f7e3c0716e0 2024-01-21T22:52:37.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.306+0000 7f7e43600640 1 -- 172.21.15.138:0/1985264569 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e3c0716e0 msgr2=0x7f7e3c0c6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:37.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.306+0000 7f7e43600640 1 --2- 172.21.15.138:0/1985264569 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e3c0716e0 0x7f7e3c0c6fe0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009870 tx=0x7f7e2402b8f0 comp rx=0 tx=0).stop 2024-01-21T22:52:37.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.306+0000 7f7e43600640 1 -- 172.21.15.138:0/1985264569 shutdown_connections 2024-01-21T22:52:37.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.306+0000 7f7e43600640 1 --2- 172.21.15.138:0/1985264569 >> v2:172.21.15.138:3300/0 conn(0x7f7e3c0c7520 0x7f7e3c06ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:37.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.306+0000 7f7e43600640 1 --2- 172.21.15.138:0/1985264569 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e3c0716e0 0x7f7e3c0c6fe0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:37.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.306+0000 7f7e43600640 1 -- 172.21.15.138:0/1985264569 >> 172.21.15.138:0/1985264569 conn(0x7f7e3c0cb0b0 msgr2=0x7f7e3c0cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:37.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.306+0000 7f7e43600640 1 -- 172.21.15.138:0/1985264569 shutdown_connections 2024-01-21T22:52:37.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.306+0000 7f7e43600640 1 -- 172.21.15.138:0/1985264569 wait complete. 2024-01-21T22:52:37.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.309+0000 7f7e43600640 1 Processor -- start 2024-01-21T22:52:37.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.309+0000 7f7e43600640 1 -- start start 2024-01-21T22:52:37.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.310+0000 7f7e43600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e3c0716e0 0x7f7e3c141650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:37.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.310+0000 7f7e43600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e3c0c7520 0x7f7e3c14e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:37.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.310+0000 7f7e43600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e3c150a00 0x7f7e3c13ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:37.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.310+0000 7f7e43600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c1ef570 con 0x7f7e3c0c7520 2024-01-21T22:52:37.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.310+0000 7f7e43600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c14f340 con 0x7f7e3c0716e0 2024-01-21T22:52:37.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.310+0000 7f7e43600640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c0c9e60 con 0x7f7e3c150a00 2024-01-21T22:52:37.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.310+0000 7f7e3be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e3c0c7520 0x7f7e3c14e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:37.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.310+0000 7f7e40c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e3c0716e0 0x7f7e3c141650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:37.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.311+0000 7f7e41600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e3c150a00 0x7f7e3c13ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:37.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.311+0000 7f7e3be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e3c0c7520 0x7f7e3c14e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60688/0 (socket says 172.21.15.138:60688) 2024-01-21T22:52:37.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.311+0000 7f7e3be00640 1 -- 172.21.15.138:0/3844893657 learned_addr learned my addr 172.21.15.138:0/3844893657 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:37.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.315+0000 7f7e3be00640 1 -- 172.21.15.138:0/3844893657 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e3c150a00 msgr2=0x7f7e3c13ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:37.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.315+0000 7f7e3be00640 1 --2- 172.21.15.138:0/3844893657 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e3c150a00 0x7f7e3c13ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:37.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.315+0000 7f7e3be00640 1 -- 172.21.15.138:0/3844893657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e3c0716e0 msgr2=0x7f7e3c141650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:37.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.315+0000 7f7e3be00640 1 --2- 172.21.15.138:0/3844893657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e3c0716e0 0x7f7e3c141650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:37.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.315+0000 7f7e3be00640 1 -- 172.21.15.138:0/3844893657 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e240094d0 con 0x7f7e3c0c7520 2024-01-21T22:52:37.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.318+0000 7f7e3be00640 1 --2- 172.21.15.138:0/3844893657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e3c0c7520 0x7f7e3c14e200 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7e24000c00 tx=0x7f7e240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:37.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.324+0000 7f7e38c00640 1 -- 172.21.15.138:0/3844893657 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2403a930 con 0x7f7e3c0c7520 2024-01-21T22:52:37.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.324+0000 7f7e38c00640 1 -- 172.21.15.138:0/3844893657 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2403aa90 con 0x7f7e3c0c7520 2024-01-21T22:52:37.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.324+0000 7f7e38c00640 1 -- 172.21.15.138:0/3844893657 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2403ae40 con 0x7f7e3c0c7520 2024-01-21T22:52:37.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.324+0000 7f7e43600640 1 -- 172.21.15.138:0/3844893657 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3c0c98a0 con 0x7f7e3c0c7520 2024-01-21T22:52:37.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.324+0000 7f7e43600640 1 -- 172.21.15.138:0/3844893657 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3c14fdc0 con 0x7f7e3c0c7520 2024-01-21T22:52:37.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.325+0000 7f7e43600640 1 -- 172.21.15.138:0/3844893657 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e3c14eda0 con 0x7f7e3c0c7520 2024-01-21T22:52:37.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.632+0000 7f7e38c00640 1 -- 172.21.15.138:0/3844893657 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f7e2402f050 con 0x7f7e3c0c7520 2024-01-21T22:52:37.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.634+0000 7f7e38c00640 1 --2- 172.21.15.138:0/3844893657 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7e1406b3b0 0x7f7e1406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:37.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.635+0000 7f7e40c00640 1 --2- 172.21.15.138:0/3844893657 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7e1406b3b0 0x7f7e1406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:37.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.636+0000 7f7e40c00640 1 --2- 172.21.15.138:0/3844893657 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7e1406b3b0 0x7f7e1406d870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7e340012e0 tx=0x7f7e34009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:37.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.659+0000 7f7e38c00640 1 -- 172.21.15.138:0/3844893657 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7e2407a5a0 con 0x7f7e3c0c7520 2024-01-21T22:52:37.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.659+0000 7f7e38c00640 1 -- 172.21.15.138:0/3844893657 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f7e240768e0 con 0x7f7e3c0c7520 2024-01-21T22:52:37.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.888+0000 7f7e43600640 1 -- 172.21.15.138:0/3844893657 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e3c14eda0 con 0x7f7e3c0c7520 2024-01-21T22:52:37.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.938+0000 7f7e38c00640 1 -- 172.21.15.138:0/3844893657 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16570 (secure 0 0 0) 0x7f7e3c14eda0 con 0x7f7e3c0c7520 2024-01-21T22:52:37.941 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:52:37.941 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":25,"fsid":"d488d446-149f-48d0-89ca-b2830ae72846","created":"2024-01-21T22:24:28.581932+0000","modified":"2024-01-21T22:25:37.558325+0000","last_up_change":"2024-01-21T22:24:47.145799+0000","last_in_change":"2024-01-21T22:24:37.672807+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-21T22:24:49.586178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-21T22:25:02.044454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-21T22:25:03.361765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d812b2d6-c8a3-4015-bfdb-5766b25eae5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6825","nonce":2849203277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6827","nonce":2849203277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6831","nonce":2849203277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6829","nonce":2849203277}]},"public_addr":"172.21.15.138:6825/2849203277","cluster_addr":"172.21.15.138:6827/2849203277","heartbeat_back_addr":"172.21.15.138:6831/2849203277","heartbeat_front_addr":"172.21.15.138:6829/2849203277","state":["exists","up"]},{"osd":1,"uuid":"d8d875df-c840-46cc-b6c0-a847f9c93999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6817","nonce":174521731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6819","nonce":174521731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6823","nonce":174521731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6821","nonce":174521731}]},"public_addr":"172.21.15.138:6817/174521731","cluster_addr":"172.21.15.138:6819/174521731","heartbeat_back_addr":"172.21.15.138:6823/174521731","heartbeat_front_addr":"172.21.15.138:6821/174521731","state":["exists","up"]},{"osd":2,"uuid":"4bd94373-76ca-47a6-870b-16d96d7c9e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6801","nonce":1365838748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6803","nonce":1365838748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6809","nonce":1365838748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6805","nonce":1365838748}]},"public_addr":"172.21.15.138:6801/1365838748","cluster_addr":"172.21.15.138:6803/1365838748","heartbeat_back_addr":"172.21.15.138:6809/1365838748","heartbeat_front_addr":"172.21.15.138:6805/1365838748","state":["exists","up"]},{"osd":3,"uuid":"07267087-d4d2-4e88-ab9b-bb87f6dd061a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6808","nonce":1841873885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6811","nonce":1841873885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6815","nonce":1841873885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6813","nonce":1841873885}]},"public_addr":"172.21.15.138:6808/1841873885","cluster_addr":"172.21.15.138:6811/1841873885","heartbeat_back_addr":"172.21.15.138:6815/1841873885","heartbeat_front_addr":"172.21.15.138:6813/1841873885","state":["exists","up"]},{"osd":4,"uuid":"b47ef758-d783-4168-a9a5-11db3de4ca12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6827","nonce":3960812298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6829","nonce":3960812298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6833","nonce":3960812298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6831","nonce":3960812298}]},"public_addr":"172.21.15.152:6827/3960812298","cluster_addr":"172.21.15.152:6829/3960812298","heartbeat_back_addr":"172.21.15.152:6833/3960812298","heartbeat_front_addr":"172.21.15.152:6831/3960812298","state":["exists","up"]},{"osd":5,"uuid":"0ea5d43e-61d8-4823-815f-9e42c62fae97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6819","nonce":4095234314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6821","nonce":4095234314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6825","nonce":4095234314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6823","nonce":4095234314}]},"public_addr":"172.21.15.152:6819/4095234314","cluster_addr":"172.21.15.152:6821/4095234314","heartbeat_back_addr":"172.21.15.152:6825/4095234314","heartbeat_front_addr":"172.21.15.152:6823/4095234314","state":["exists","up"]},{"osd":6,"uuid":"59ffac36-e2e0-4693-ac82-ad5eccf6224d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6803","nonce":223848127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6805","nonce":223848127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6809","nonce":223848127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6807","nonce":223848127}]},"public_addr":"172.21.15.152:6803/223848127","cluster_addr":"172.21.15.152:6805/223848127","heartbeat_back_addr":"172.21.15.152:6809/223848127","heartbeat_front_addr":"172.21.15.152:6807/223848127","state":["exists","up"]},{"osd":7,"uuid":"4f892c7d-32a5-4960-bf88-4ed7c863cd0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6811","nonce":52509095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6813","nonce":52509095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6817","nonce":52509095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6815","nonce":52509095}]},"public_addr":"172.21.15.152:6811/52509095","cluster_addr":"172.21.15.152:6813/52509095","heartbeat_back_addr":"172.21.15.152:6817/52509095","heartbeat_front_addr":"172.21.15.152:6815/52509095","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.434460+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.374709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.613134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.298148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.242144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.198102+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.093385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.197180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-21T22:52:37.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.942+0000 7f7e43600640 1 -- 172.21.15.138:0/3844893657 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7e1406b3b0 msgr2=0x7f7e1406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:37.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.942+0000 7f7e43600640 1 --2- 172.21.15.138:0/3844893657 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7e1406b3b0 0x7f7e1406d870 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7e340012e0 tx=0x7f7e34009040 comp rx=0 tx=0).stop 2024-01-21T22:52:37.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.942+0000 7f7e43600640 1 -- 172.21.15.138:0/3844893657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e3c0c7520 msgr2=0x7f7e3c14e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:37.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.942+0000 7f7e43600640 1 --2- 172.21.15.138:0/3844893657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e3c0c7520 0x7f7e3c14e200 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7e24000c00 tx=0x7f7e240046c0 comp rx=0 tx=0).stop 2024-01-21T22:52:37.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.942+0000 7f7e43600640 1 -- 172.21.15.138:0/3844893657 shutdown_connections 2024-01-21T22:52:37.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.942+0000 7f7e43600640 1 --2- 172.21.15.138:0/3844893657 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7e1406b3b0 0x7f7e1406d870 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:37.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.942+0000 7f7e43600640 1 --2- 172.21.15.138:0/3844893657 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e3c150a00 0x7f7e3c13ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:37.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.942+0000 7f7e43600640 1 --2- 172.21.15.138:0/3844893657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e3c0c7520 0x7f7e3c14e200 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:37.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.942+0000 7f7e43600640 1 --2- 172.21.15.138:0/3844893657 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e3c0716e0 0x7f7e3c141650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:37.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.942+0000 7f7e43600640 1 -- 172.21.15.138:0/3844893657 >> 172.21.15.138:0/3844893657 conn(0x7f7e3c0cb0b0 msgr2=0x7f7e3c0c24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:37.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.943+0000 7f7e43600640 1 -- 172.21.15.138:0/3844893657 shutdown_connections 2024-01-21T22:52:37.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:37.943+0000 7f7e43600640 1 -- 172.21.15.138:0/3844893657 wait complete. 2024-01-21T22:52:37.963 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-01-21T22:24:49.586178+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-01-21T22:25:02.044454+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-01-21T22:25:03.361765+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-01-21T22:52:37.964 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-01-21T22:52:38.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.154+0000 7f22b8600640 1 Processor -- start 2024-01-21T22:52:38.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.154+0000 7f22b8600640 1 -- start start 2024-01-21T22:52:38.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.156+0000 7f22b8600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f22b006b4f0 0x7f22b00c4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:38.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.156+0000 7f22b8600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f22b0071bb0 0x7f22b00c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:38.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.156+0000 7f22b8600640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b0070590 con 0x7f22b0151a30 2024-01-21T22:52:38.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.156+0000 7f22b8600640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b006ac50 con 0x7f22b006b4f0 2024-01-21T22:52:38.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.156+0000 7f22b8600640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b006a6f0 con 0x7f22b0071bb0 2024-01-21T22:52:38.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.157+0000 7f22b6600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f22b0071bb0 0x7f22b00c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:38.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.157+0000 7f22b6600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f22b0071bb0 0x7f22b00c4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:55140/0 (socket says 172.21.15.138:55140) 2024-01-21T22:52:38.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.157+0000 7f22b5c00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f22b0151a30 0x7f22b0157730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:38328/0 (socket says 172.21.15.138:38328) 2024-01-21T22:52:38.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.157+0000 7f22b6600640 1 -- 172.21.15.138:0/1861940634 learned_addr learned my addr 172.21.15.138:0/1861940634 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:38.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.158+0000 7f22b5200640 1 --2- 172.21.15.138:0/1861940634 >> v2:172.21.15.138:3300/0 conn(0x7f22b006b4f0 0x7f22b00c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:38.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.159+0000 7f22b6600640 1 -- 172.21.15.138:0/1861940634 >> v2:172.21.15.138:3300/0 conn(0x7f22b006b4f0 msgr2=0x7f22b00c4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:38.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.159+0000 7f22b6600640 1 --2- 172.21.15.138:0/1861940634 >> v2:172.21.15.138:3300/0 conn(0x7f22b006b4f0 0x7f22b00c4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:38.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.159+0000 7f22b6600640 1 -- 172.21.15.138:0/1861940634 >> v1:172.21.15.138:6789/0 conn(0x7f22b0151a30 legacy=0x7f22b0157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:38.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.160+0000 7f22b6600640 1 -- 172.21.15.138:0/1861940634 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b0151e10 con 0x7f22b0071bb0 2024-01-21T22:52:38.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.162+0000 7f22b6600640 1 --2- 172.21.15.138:0/1861940634 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f22b0071bb0 0x7f22b00c4c90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f22a8009e50 tx=0x7f22a802bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b42a40c7b373552 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:38.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.166+0000 7f22afe00640 1 -- 172.21.15.138:0/1861940634 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a802cb90 con 0x7f22b0071bb0 2024-01-21T22:52:38.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.166+0000 7f22afe00640 1 -- 172.21.15.138:0/1861940634 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a802ccf0 con 0x7f22b0071bb0 2024-01-21T22:52:38.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.166+0000 7f22afe00640 1 -- 172.21.15.138:0/1861940634 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a8034610 con 0x7f22b0071bb0 2024-01-21T22:52:38.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.167+0000 7f22b8600640 1 -- 172.21.15.138:0/1861940634 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f22b0071bb0 msgr2=0x7f22b00c4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:38.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.167+0000 7f22b8600640 1 --2- 172.21.15.138:0/1861940634 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f22b0071bb0 0x7f22b00c4c90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f22a8009e50 tx=0x7f22a802bff0 comp rx=0 tx=0).stop 2024-01-21T22:52:38.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.167+0000 7f22b8600640 1 -- 172.21.15.138:0/1861940634 shutdown_connections 2024-01-21T22:52:38.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.167+0000 7f22b8600640 1 --2- 172.21.15.138:0/1861940634 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f22b0071bb0 0x7f22b00c4c90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:38.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.167+0000 7f22b8600640 1 --2- 172.21.15.138:0/1861940634 >> v2:172.21.15.138:3300/0 conn(0x7f22b006b4f0 0x7f22b00c4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:38.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.167+0000 7f22b8600640 1 -- 172.21.15.138:0/1861940634 >> 172.21.15.138:0/1861940634 conn(0x7f22b0072a20 msgr2=0x7f22b0066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:38.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.167+0000 7f22b8600640 1 -- 172.21.15.138:0/1861940634 shutdown_connections 2024-01-21T22:52:38.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.167+0000 7f22b8600640 1 -- 172.21.15.138:0/1861940634 wait complete. 2024-01-21T22:52:38.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.169+0000 7f22b8600640 1 Processor -- start 2024-01-21T22:52:38.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.169+0000 7f22b8600640 1 -- start start 2024-01-21T22:52:38.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.170+0000 7f22b8600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f22b006b4f0 0x7f22b00c4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:38.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.170+0000 7f22b8600640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22b0071bb0 0x7f22b01e1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:38.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.170+0000 7f22b8600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22b0151a30 0x7f22b01439c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:38.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.170+0000 7f22b8600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b00c8e40 con 0x7f22b0151a30 2024-01-21T22:52:38.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.170+0000 7f22b8600640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b01e0eb0 con 0x7f22b0071bb0 2024-01-21T22:52:38.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.170+0000 7f22b8600640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b01ef680 con 0x7f22b006b4f0 2024-01-21T22:52:38.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.171+0000 7f22b6600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22b0151a30 0x7f22b01439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:38.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.171+0000 7f22b5c00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f22b006b4f0 0x7f22b00c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:38.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.171+0000 7f22b5200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22b0071bb0 0x7f22b01e1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:38.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.171+0000 7f22b6600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22b0151a30 0x7f22b01439c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50814/0 (socket says 172.21.15.138:50814) 2024-01-21T22:52:38.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.171+0000 7f22b6600640 1 -- 172.21.15.138:0/847780327 learned_addr learned my addr 172.21.15.138:0/847780327 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:38.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.175+0000 7f22b5c00640 1 -- 172.21.15.138:0/847780327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22b0071bb0 msgr2=0x7f22b01e1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:38.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.175+0000 7f22b5c00640 1 --2- 172.21.15.138:0/847780327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22b0071bb0 0x7f22b01e1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:38.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.175+0000 7f22b5c00640 1 -- 172.21.15.138:0/847780327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22b0151a30 msgr2=0x7f22b01439c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:38.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.175+0000 7f22b5c00640 1 --2- 172.21.15.138:0/847780327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22b0151a30 0x7f22b01439c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:38.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.175+0000 7f22b5c00640 1 -- 172.21.15.138:0/847780327 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a8009b30 con 0x7f22b006b4f0 2024-01-21T22:52:38.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.177+0000 7f22b5c00640 1 --2- 172.21.15.138:0/847780327 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f22b006b4f0 0x7f22b00c4350 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f22b00c23d0 tx=0x7f22a000c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:38.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.180+0000 7f22ad600640 1 -- 172.21.15.138:0/847780327 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a00074e0 con 0x7f22b006b4f0 2024-01-21T22:52:38.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.181+0000 7f22ad600640 1 -- 172.21.15.138:0/847780327 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a0007640 con 0x7f22b006b4f0 2024-01-21T22:52:38.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.181+0000 7f22ad600640 1 -- 172.21.15.138:0/847780327 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0004a00 con 0x7f22b006b4f0 2024-01-21T22:52:38.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.181+0000 7f22b8600640 1 -- 172.21.15.138:0/847780327 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b01ef940 con 0x7f22b006b4f0 2024-01-21T22:52:38.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.181+0000 7f22b8600640 1 -- 172.21.15.138:0/847780327 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b01eff00 con 0x7f22b006b4f0 2024-01-21T22:52:38.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.182+0000 7f22b8600640 1 -- 172.21.15.138:0/847780327 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b004eb10 con 0x7f22b006b4f0 2024-01-21T22:52:38.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.203+0000 7f22ad600640 1 -- 172.21.15.138:0/847780327 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f22a0004b60 con 0x7f22b006b4f0 2024-01-21T22:52:38.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.204+0000 7f22ad600640 1 --2- 172.21.15.138:0/847780327 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f228806b3b0 0x7f228806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:38.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.204+0000 7f22b5200640 1 --2- 172.21.15.138:0/847780327 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f228806b3b0 0x7f228806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:38.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.204+0000 7f22ad600640 1 -- 172.21.15.138:0/847780327 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f22a00584c0 con 0x7f22b006b4f0 2024-01-21T22:52:38.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.205+0000 7f22b5200640 1 --2- 172.21.15.138:0/847780327 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f228806b3b0 0x7f228806d870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f22b014c760 tx=0x7f2298009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:38.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.455+0000 7f22ad600640 1 -- 172.21.15.138:0/847780327 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f22a0054800 con 0x7f22b006b4f0 2024-01-21T22:52:38.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.654+0000 7f22b8600640 1 -- 172.21.15.138:0/847780327 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f22b014e740 con 0x7f22b006b4f0 2024-01-21T22:52:38.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.660+0000 7f22ad600640 1 -- 172.21.15.138:0/847780327 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f22a0057e20 con 0x7f22b006b4f0 2024-01-21T22:52:38.661 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-01-21T22:52:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.662+0000 7f22b8600640 1 -- 172.21.15.138:0/847780327 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f228806b3b0 msgr2=0x7f228806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.662+0000 7f22b8600640 1 --2- 172.21.15.138:0/847780327 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f228806b3b0 0x7f228806d870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f22b014c760 tx=0x7f2298009190 comp rx=0 tx=0).stop 2024-01-21T22:52:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.662+0000 7f22b8600640 1 -- 172.21.15.138:0/847780327 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f22b006b4f0 msgr2=0x7f22b00c4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.662+0000 7f22b8600640 1 --2- 172.21.15.138:0/847780327 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f22b006b4f0 0x7f22b00c4350 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f22b00c23d0 tx=0x7f22a000c760 comp rx=0 tx=0).stop 2024-01-21T22:52:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.662+0000 7f22b8600640 1 -- 172.21.15.138:0/847780327 shutdown_connections 2024-01-21T22:52:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.662+0000 7f22b8600640 1 --2- 172.21.15.138:0/847780327 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f228806b3b0 0x7f228806d870 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:38.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.662+0000 7f22b8600640 1 --2- 172.21.15.138:0/847780327 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f22b0151a30 0x7f22b01439c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:38.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.662+0000 7f22b8600640 1 --2- 172.21.15.138:0/847780327 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f22b0071bb0 0x7f22b01e1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:38.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.662+0000 7f22b8600640 1 --2- 172.21.15.138:0/847780327 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f22b006b4f0 0x7f22b00c4350 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:38.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.662+0000 7f22b8600640 1 -- 172.21.15.138:0/847780327 >> 172.21.15.138:0/847780327 conn(0x7f22b0072a20 msgr2=0x7f22b01e5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:38.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.663+0000 7f22b8600640 1 -- 172.21.15.138:0/847780327 shutdown_connections 2024-01-21T22:52:38.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.663+0000 7f22b8600640 1 -- 172.21.15.138:0/847780327 wait complete. 2024-01-21T22:52:38.683 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-01-21T22:52:38.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.891+0000 7fd032400640 1 Processor -- start 2024-01-21T22:52:38.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.891+0000 7fd032400640 1 -- start start 2024-01-21T22:52:38.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.893+0000 7fd032400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd02c06b600 0x7fd02c15b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:38.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.893+0000 7fd032400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd02c070a80 con 0x7fd02c06d190 2024-01-21T22:52:38.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.893+0000 7fd032400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd02c151e60 con 0x7fd02c151a80 2024-01-21T22:52:38.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.893+0000 7fd032400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd02c072c70 con 0x7fd02c06b600 2024-01-21T22:52:38.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.894+0000 7fd031000640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd02c151a80 0x7fd02c06aef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:60430/0 (socket says 172.21.15.138:60430) 2024-01-21T22:52:38.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.894+0000 7fd031000640 1 -- 172.21.15.138:0/3059096209 learned_addr learned my addr 172.21.15.138:0/3059096209 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:52:38.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.894+0000 7fd031a00640 1 --2- 172.21.15.138:0/3059096209 >> v2:172.21.15.138:3300/0 conn(0x7fd02c06b600 0x7fd02c15b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:38.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.896+0000 7fd02be00640 1 -- 172.21.15.138:0/3059096209 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3242693593 0 0) 0x7fd02c151e60 con 0x7fd02c151a80 2024-01-21T22:52:38.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.896+0000 7fd02be00640 1 -- 172.21.15.138:0/3059096209 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd00c003520 con 0x7fd02c151a80 2024-01-21T22:52:38.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.900+0000 7fd02be00640 1 -- 172.21.15.138:0/3059096209 <== mon.1 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 187453638 0 0) 0x7fd00c003520 con 0x7fd02c151a80 2024-01-21T22:52:38.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.900+0000 7fd02be00640 1 -- 172.21.15.138:0/3059096209 >> v2:172.21.15.138:3300/0 conn(0x7fd02c06b600 msgr2=0x7fd02c15b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:38.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.900+0000 7fd02be00640 1 --2- 172.21.15.138:0/3059096209 >> v2:172.21.15.138:3300/0 conn(0x7fd02c06b600 0x7fd02c15b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:38.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.900+0000 7fd02be00640 1 -- 172.21.15.138:0/3059096209 >> v1:172.21.15.138:6789/0 conn(0x7fd02c06d190 legacy=0x7fd02c15ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:38.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.900+0000 7fd02be00640 1 -- 172.21.15.138:0/3059096209 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02c070d20 con 0x7fd02c151a80 2024-01-21T22:52:38.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.900+0000 7fd02be00640 1 -- 172.21.15.138:0/3059096209 <== mon.1 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1288616872 0 0) 0x7fd01c001670 con 0x7fd02c151a80 2024-01-21T22:52:38.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.900+0000 7fd02be00640 1 -- 172.21.15.138:0/3059096209 >> v1:172.21.15.152:6789/0 conn(0x7fd02c151a80 legacy=0x7fd02c06aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:38.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.900+0000 7fd02be00640 1 --2- 172.21.15.138:0/3059096209 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd00c0039c0 0x7fd00c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:38.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.900+0000 7fd02be00640 1 --2- 172.21.15.138:0/3059096209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd00c0047c0 0x7fd00c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:38.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.900+0000 7fd02be00640 1 --2- 172.21.15.138:0/3059096209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd00c005180 0x7fd00c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:38.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.900+0000 7fd02be00640 1 -- 172.21.15.138:0/3059096209 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c072c70 con 0x7fd00c005180 2024-01-21T22:52:38.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.900+0000 7fd02be00640 1 -- 172.21.15.138:0/3059096209 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c00c060 con 0x7fd00c0047c0 2024-01-21T22:52:38.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.900+0000 7fd02be00640 1 -- 172.21.15.138:0/3059096209 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c00c220 con 0x7fd00c0039c0 2024-01-21T22:52:38.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.901+0000 7fd031000640 1 --2- 172.21.15.138:0/3059096209 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd00c0039c0 0x7fd00c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:38.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.903+0000 7fd023e00640 1 --2- 172.21.15.138:0/3059096209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd00c0047c0 0x7fd00c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:38.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.904+0000 7fd031000640 1 -- 172.21.15.138:0/3059096209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd00c0047c0 msgr2=0x7fd00c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:38.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.904+0000 7fd031000640 1 --2- 172.21.15.138:0/3059096209 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd00c0047c0 0x7fd00c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:38.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.904+0000 7fd031000640 1 -- 172.21.15.138:0/3059096209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd00c005180 msgr2=0x7fd00c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:38.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.904+0000 7fd031000640 1 --2- 172.21.15.138:0/3059096209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd00c005180 0x7fd00c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:38.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.904+0000 7fd031000640 1 -- 172.21.15.138:0/3059096209 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00c003520 con 0x7fd00c0039c0 2024-01-21T22:52:38.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.904+0000 7fd031a00640 1 -- 172.21.15.138:0/3059096209 reap_dead start 2024-01-21T22:52:38.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.906+0000 7fd031000640 1 --2- 172.21.15.138:0/3059096209 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd00c0039c0 0x7fd00c003de0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00c0e0 tx=0x7fd00c0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3002078aeaac750 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:38.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.910+0000 7fd02be00640 1 -- 172.21.15.138:0/3059096209 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c005ad0 con 0x7fd00c0039c0 2024-01-21T22:52:38.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.910+0000 7fd02be00640 1 -- 172.21.15.138:0/3059096209 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01c005c30 con 0x7fd00c0039c0 2024-01-21T22:52:38.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.910+0000 7fd02be00640 1 -- 172.21.15.138:0/3059096209 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01c02c7d0 con 0x7fd00c0039c0 2024-01-21T22:52:38.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.911+0000 7fd032400640 1 -- 172.21.15.138:0/3059096209 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd00c0039c0 msgr2=0x7fd00c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:38.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.911+0000 7fd032400640 1 --2- 172.21.15.138:0/3059096209 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd00c0039c0 0x7fd00c003de0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd01c00c0e0 tx=0x7fd00c0057f0 comp rx=0 tx=0).stop 2024-01-21T22:52:38.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.911+0000 7fd032400640 1 -- 172.21.15.138:0/3059096209 shutdown_connections 2024-01-21T22:52:38.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.911+0000 7fd032400640 1 --2- 172.21.15.138:0/3059096209 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd00c0039c0 0x7fd00c003de0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:38.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.911+0000 7fd032400640 1 -- 172.21.15.138:0/3059096209 >> 172.21.15.138:0/3059096209 conn(0x7fd02c06d9e0 msgr2=0x7fd02c06ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:38.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.911+0000 7fd032400640 1 -- 172.21.15.138:0/3059096209 shutdown_connections 2024-01-21T22:52:38.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.911+0000 7fd032400640 1 -- 172.21.15.138:0/3059096209 wait complete. 2024-01-21T22:52:38.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.913+0000 7fd032400640 1 Processor -- start 2024-01-21T22:52:38.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.914+0000 7fd032400640 1 -- start start 2024-01-21T22:52:38.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.915+0000 7fd032400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd00c0039c0 0x7fd02c0c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:38.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.915+0000 7fd032400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd02c0c2d10 0x7fd02c158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:38.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.915+0000 7fd032400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd02c158800 0x7fd02c0aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:38.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.915+0000 7fd032400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c0c7890 con 0x7fd02c158800 2024-01-21T22:52:38.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.915+0000 7fd032400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c0c45c0 con 0x7fd02c0c2d10 2024-01-21T22:52:38.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.915+0000 7fd032400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02c0c72d0 con 0x7fd00c0039c0 2024-01-21T22:52:38.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.916+0000 7fd031a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd02c158800 0x7fd02c0aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:38.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.916+0000 7fd023e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd02c0c2d10 0x7fd02c158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:38.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.916+0000 7fd031000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd00c0039c0 0x7fd02c0c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:38.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.916+0000 7fd031a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd02c158800 0x7fd02c0aeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50836/0 (socket says 172.21.15.138:50836) 2024-01-21T22:52:38.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.916+0000 7fd031a00640 1 -- 172.21.15.138:0/2776631661 learned_addr learned my addr 172.21.15.138:0/2776631661 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:38.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.920+0000 7fd023e00640 1 -- 172.21.15.138:0/2776631661 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd00c0039c0 msgr2=0x7fd02c0c4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:38.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.920+0000 7fd023e00640 1 --2- 172.21.15.138:0/2776631661 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd00c0039c0 0x7fd02c0c4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:38.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.920+0000 7fd023e00640 1 -- 172.21.15.138:0/2776631661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd02c158800 msgr2=0x7fd02c0aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:38.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.920+0000 7fd023e00640 1 --2- 172.21.15.138:0/2776631661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd02c158800 0x7fd02c0aeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:38.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.920+0000 7fd023e00640 1 -- 172.21.15.138:0/2776631661 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd02c0c75b0 con 0x7fd02c0c2d10 2024-01-21T22:52:38.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.923+0000 7fd023e00640 1 --2- 172.21.15.138:0/2776631661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd02c0c2d10 0x7fd02c158e80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd01400e3f0 tx=0x7fd01400e8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:38.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.926+0000 7fd029600640 1 -- 172.21.15.138:0/2776631661 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0140052e0 con 0x7fd02c0c2d10 2024-01-21T22:52:38.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.926+0000 7fd029600640 1 -- 172.21.15.138:0/2776631661 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd014005440 con 0x7fd02c0c2d10 2024-01-21T22:52:38.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.926+0000 7fd029600640 1 -- 172.21.15.138:0/2776631661 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd014003810 con 0x7fd02c0c2d10 2024-01-21T22:52:38.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.926+0000 7fd032400640 1 -- 172.21.15.138:0/2776631661 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd02c0c3f70 con 0x7fd02c0c2d10 2024-01-21T22:52:38.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.927+0000 7fd032400640 1 -- 172.21.15.138:0/2776631661 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd02c0af970 con 0x7fd02c0c2d10 2024-01-21T22:52:38.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.928+0000 7fd032400640 1 -- 172.21.15.138:0/2776631661 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd02c0c3dd0 con 0x7fd02c0c2d10 2024-01-21T22:52:38.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.947+0000 7fd029600640 1 -- 172.21.15.138:0/2776631661 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fd014003970 con 0x7fd02c0c2d10 2024-01-21T22:52:38.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.949+0000 7fd029600640 1 --2- 172.21.15.138:0/2776631661 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd00406b3b0 0x7fd00406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:38.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.949+0000 7fd031000640 1 --2- 172.21.15.138:0/2776631661 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd00406b3b0 0x7fd00406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:38.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.950+0000 7fd029600640 1 -- 172.21.15.138:0/2776631661 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd014058440 con 0x7fd02c0c2d10 2024-01-21T22:52:38.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:38.950+0000 7fd031000640 1 --2- 172.21.15.138:0/2776631661 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd00406b3b0 0x7fd00406d870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd01c034000 tx=0x7fd01c009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:39.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.190+0000 7fd029600640 1 -- 172.21.15.138:0/2776631661 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd014054780 con 0x7fd02c0c2d10 2024-01-21T22:52:39.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.404+0000 7fd032400640 1 -- 172.21.15.138:0/2776631661 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd02c0c3dd0 con 0x7fd02c0c2d10 2024-01-21T22:52:39.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.457+0000 7fd029600640 1 -- 172.21.15.138:0/2776631661 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd02c0c3dd0 con 0x7fd02c0c2d10 2024-01-21T22:52:39.459 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 32 2024-01-21T22:52:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.460+0000 7fd032400640 1 -- 172.21.15.138:0/2776631661 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd00406b3b0 msgr2=0x7fd00406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.460+0000 7fd032400640 1 --2- 172.21.15.138:0/2776631661 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd00406b3b0 0x7fd00406d870 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd01c034000 tx=0x7fd01c009030 comp rx=0 tx=0).stop 2024-01-21T22:52:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.460+0000 7fd032400640 1 -- 172.21.15.138:0/2776631661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd02c0c2d10 msgr2=0x7fd02c158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.460+0000 7fd032400640 1 --2- 172.21.15.138:0/2776631661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd02c0c2d10 0x7fd02c158e80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd01400e3f0 tx=0x7fd01400e8b0 comp rx=0 tx=0).stop 2024-01-21T22:52:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.460+0000 7fd032400640 1 -- 172.21.15.138:0/2776631661 shutdown_connections 2024-01-21T22:52:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.460+0000 7fd032400640 1 --2- 172.21.15.138:0/2776631661 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd00406b3b0 0x7fd00406d870 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.460+0000 7fd032400640 1 --2- 172.21.15.138:0/2776631661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd02c158800 0x7fd02c0aeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:39.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.460+0000 7fd032400640 1 --2- 172.21.15.138:0/2776631661 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd02c0c2d10 0x7fd02c158e80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:39.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.460+0000 7fd032400640 1 --2- 172.21.15.138:0/2776631661 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd00c0039c0 0x7fd02c0c4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:39.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.460+0000 7fd032400640 1 -- 172.21.15.138:0/2776631661 >> 172.21.15.138:0/2776631661 conn(0x7fd02c06d9e0 msgr2=0x7fd02c14d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:39.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.461+0000 7fd032400640 1 -- 172.21.15.138:0/2776631661 shutdown_connections 2024-01-21T22:52:39.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.461+0000 7fd032400640 1 -- 172.21.15.138:0/2776631661 wait complete. 2024-01-21T22:52:39.485 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-01-21T22:52:39.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.691+0000 7f186f400640 1 Processor -- start 2024-01-21T22:52:39.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.691+0000 7f186f400640 1 -- start start 2024-01-21T22:52:39.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.692+0000 7f186f400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18680c8b40 0x7f1868165590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:39.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.692+0000 7f186f400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18680c72d0 con 0x7f186806eca0 2024-01-21T22:52:39.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.692+0000 7f186f400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f186806b8a0 con 0x7f186815a900 2024-01-21T22:52:39.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.692+0000 7f186f400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18680c6fa0 con 0x7f18680c8b40 2024-01-21T22:52:39.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.693+0000 7f1867e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f186806eca0 0x7f18680c8430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:38350/0 (socket says 172.21.15.138:38350) 2024-01-21T22:52:39.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.693+0000 7f1867e00640 1 -- 172.21.15.138:0/3289347059 learned_addr learned my addr 172.21.15.138:0/3289347059 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:52:39.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.694+0000 7f186d400640 1 --2- 172.21.15.138:0/3289347059 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18680c8b40 0x7f1868165590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:39.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.695+0000 7f1867400640 1 -- 172.21.15.138:0/3289347059 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936550754 0 0) 0x7f186806b8a0 con 0x7f186815a900 2024-01-21T22:52:39.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.696+0000 7f1867400640 1 -- 172.21.15.138:0/3289347059 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1848003520 con 0x7f186815a900 2024-01-21T22:52:39.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.696+0000 7f1867400640 1 -- 172.21.15.138:0/3289347059 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 353621293 0 0) 0x7f18680c72d0 con 0x7f186806eca0 2024-01-21T22:52:39.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.696+0000 7f1867400640 1 -- 172.21.15.138:0/3289347059 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f186806b8a0 con 0x7f186806eca0 2024-01-21T22:52:39.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.696+0000 7f186d400640 1 -- 172.21.15.138:0/3289347059 >> v1:172.21.15.152:6789/0 conn(0x7f186815a900 legacy=0x7f18680cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:39.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.696+0000 7f186d400640 1 -- 172.21.15.138:0/3289347059 >> v1:172.21.15.138:6789/0 conn(0x7f186806eca0 legacy=0x7f18680c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:39.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.696+0000 7f186d400640 1 -- 172.21.15.138:0/3289347059 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186806bb40 con 0x7f18680c8b40 2024-01-21T22:52:39.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.698+0000 7f186d400640 1 --2- 172.21.15.138:0/3289347059 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18680c8b40 0x7f1868165590 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f18600097d0 tx=0x7f186002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beb3756cf4f9894f server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:39.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.702+0000 7f1867400640 1 -- 172.21.15.138:0/3289347059 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1860031890 con 0x7f18680c8b40 2024-01-21T22:52:39.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.702+0000 7f1867400640 1 -- 172.21.15.138:0/3289347059 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18600319f0 con 0x7f18680c8b40 2024-01-21T22:52:39.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.702+0000 7f1867400640 1 -- 172.21.15.138:0/3289347059 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1860031d80 con 0x7f18680c8b40 2024-01-21T22:52:39.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.703+0000 7f186f400640 1 -- 172.21.15.138:0/3289347059 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18680c8b40 msgr2=0x7f1868165590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:39.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.703+0000 7f186f400640 1 --2- 172.21.15.138:0/3289347059 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18680c8b40 0x7f1868165590 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f18600097d0 tx=0x7f186002b930 comp rx=0 tx=0).stop 2024-01-21T22:52:39.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.703+0000 7f186f400640 1 -- 172.21.15.138:0/3289347059 shutdown_connections 2024-01-21T22:52:39.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.703+0000 7f186f400640 1 --2- 172.21.15.138:0/3289347059 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18680c8b40 0x7f1868165590 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:39.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.703+0000 7f186f400640 1 -- 172.21.15.138:0/3289347059 >> 172.21.15.138:0/3289347059 conn(0x7f186806a8d0 msgr2=0x7f1868066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:39.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.703+0000 7f186f400640 1 -- 172.21.15.138:0/3289347059 shutdown_connections 2024-01-21T22:52:39.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.703+0000 7f186f400640 1 -- 172.21.15.138:0/3289347059 wait complete. 2024-01-21T22:52:39.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.705+0000 7f186f400640 1 Processor -- start 2024-01-21T22:52:39.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.705+0000 7f186f400640 1 -- start start 2024-01-21T22:52:39.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.706+0000 7f186f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f186806eca0 0x7f18680cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:39.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.706+0000 7f186f400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18680c8b40 0x7f18681f0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:39.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.706+0000 7f186f400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f186815a900 0x7f18680c3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:39.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.706+0000 7f186f400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18680c6700 con 0x7f186806eca0 2024-01-21T22:52:39.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.706+0000 7f186f400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1868156b00 con 0x7f186815a900 2024-01-21T22:52:39.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.706+0000 7f186f400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1868156f90 con 0x7f18680c8b40 2024-01-21T22:52:39.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.707+0000 7f186ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f186806eca0 0x7f18680cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:39.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.707+0000 7f186d400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f186815a900 0x7f18680c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:39.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.707+0000 7f1867e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18680c8b40 0x7f18681f0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:39.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.707+0000 7f186ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f186806eca0 0x7f18680cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50840/0 (socket says 172.21.15.138:50840) 2024-01-21T22:52:39.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.707+0000 7f186ca00640 1 -- 172.21.15.138:0/557736095 learned_addr learned my addr 172.21.15.138:0/557736095 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:39.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.710+0000 7f1867e00640 1 -- 172.21.15.138:0/557736095 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f186815a900 msgr2=0x7f18680c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:39.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.710+0000 7f1867e00640 1 --2- 172.21.15.138:0/557736095 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f186815a900 0x7f18680c3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:39.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.710+0000 7f1867e00640 1 -- 172.21.15.138:0/557736095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f186806eca0 msgr2=0x7f18680cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:39.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.710+0000 7f1867e00640 1 --2- 172.21.15.138:0/557736095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f186806eca0 0x7f18680cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:39.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.710+0000 7f1867e00640 1 -- 172.21.15.138:0/557736095 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1860009400 con 0x7f18680c8b40 2024-01-21T22:52:39.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.710+0000 7f186d400640 1 --2- 172.21.15.138:0/557736095 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f186815a900 0x7f18680c3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-21T22:52:39.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.713+0000 7f1867e00640 1 --2- 172.21.15.138:0/557736095 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18680c8b40 0x7f18681f0850 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1868072460 tx=0x7f185000c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:39.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.717+0000 7f1864c00640 1 -- 172.21.15.138:0/557736095 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1850005ce0 con 0x7f18680c8b40 2024-01-21T22:52:39.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.717+0000 7f1864c00640 1 -- 172.21.15.138:0/557736095 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1850005e40 con 0x7f18680c8b40 2024-01-21T22:52:39.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.717+0000 7f1864c00640 1 -- 172.21.15.138:0/557736095 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18500072f0 con 0x7f18680c8b40 2024-01-21T22:52:39.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.717+0000 7f186f400640 1 -- 172.21.15.138:0/557736095 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18680c6420 con 0x7f18680c8b40 2024-01-21T22:52:39.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.717+0000 7f186f400640 1 -- 172.21.15.138:0/557736095 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18681564b0 con 0x7f18680c8b40 2024-01-21T22:52:39.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.718+0000 7f186f400640 1 -- 172.21.15.138:0/557736095 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1868156c70 con 0x7f18680c8b40 2024-01-21T22:52:39.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.738+0000 7f1864c00640 1 -- 172.21.15.138:0/557736095 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f1850007450 con 0x7f18680c8b40 2024-01-21T22:52:39.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.739+0000 7f1864c00640 1 --2- 172.21.15.138:0/557736095 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f184006b380 0x7f184006d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:39.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.739+0000 7f186ca00640 1 --2- 172.21.15.138:0/557736095 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f184006b380 0x7f184006d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:39.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.740+0000 7f186ca00640 1 --2- 172.21.15.138:0/557736095 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f184006b380 0x7f184006d840 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1858001ed0 tx=0x7f1858009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:39.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.967+0000 7f1864c00640 1 -- 172.21.15.138:0/557736095 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1850007e40 con 0x7f18680c8b40 2024-01-21T22:52:39.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:39.977+0000 7f1864c00640 1 -- 172.21.15.138:0/557736095 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f1850054250 con 0x7f18680c8b40 2024-01-21T22:52:40.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.177+0000 7f186f400640 1 -- 172.21.15.138:0/557736095 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f186806ea80 con 0x7f18680c8b40 2024-01-21T22:52:40.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.184+0000 7f1864c00640 1 -- 172.21.15.138:0/557736095 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1850017020 con 0x7f18680c8b40 2024-01-21T22:52:40.185 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 64 2024-01-21T22:52:40.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.186+0000 7f186f400640 1 -- 172.21.15.138:0/557736095 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f184006b380 msgr2=0x7f184006d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:40.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.186+0000 7f186f400640 1 --2- 172.21.15.138:0/557736095 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f184006b380 0x7f184006d840 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1858001ed0 tx=0x7f1858009040 comp rx=0 tx=0).stop 2024-01-21T22:52:40.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.186+0000 7f186f400640 1 -- 172.21.15.138:0/557736095 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18680c8b40 msgr2=0x7f18681f0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:40.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.186+0000 7f186f400640 1 --2- 172.21.15.138:0/557736095 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18680c8b40 0x7f18681f0850 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1868072460 tx=0x7f185000c890 comp rx=0 tx=0).stop 2024-01-21T22:52:40.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.186+0000 7f186f400640 1 -- 172.21.15.138:0/557736095 shutdown_connections 2024-01-21T22:52:40.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.186+0000 7f186f400640 1 --2- 172.21.15.138:0/557736095 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f184006b380 0x7f184006d840 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:40.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.186+0000 7f186f400640 1 --2- 172.21.15.138:0/557736095 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f186815a900 0x7f18680c3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:40.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.186+0000 7f186f400640 1 --2- 172.21.15.138:0/557736095 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f18680c8b40 0x7f18681f0850 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:40.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.186+0000 7f186f400640 1 --2- 172.21.15.138:0/557736095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f186806eca0 0x7f18680cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:40.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.186+0000 7f186f400640 1 -- 172.21.15.138:0/557736095 >> 172.21.15.138:0/557736095 conn(0x7f186806a8d0 msgr2=0x7f186806dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:40.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.186+0000 7f186f400640 1 -- 172.21.15.138:0/557736095 shutdown_connections 2024-01-21T22:52:40.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.186+0000 7f186f400640 1 -- 172.21.15.138:0/557736095 wait complete. 2024-01-21T22:52:40.207 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-01-21T22:52:40.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.415+0000 7fcbd9400640 1 Processor -- start 2024-01-21T22:52:40.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.416+0000 7fcbd9400640 1 -- start start 2024-01-21T22:52:40.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.417+0000 7fcbd9400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcbd415cdc0 0x7fcbd40c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:40.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.417+0000 7fcbd9400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fcbd415d1a0 0x7fcbd40c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:40.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.417+0000 7fcbd9400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbd40c4900 con 0x7fcbd40ca7b0 2024-01-21T22:52:40.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.417+0000 7fcbd9400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbd40c52d0 con 0x7fcbd415d1a0 2024-01-21T22:52:40.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.417+0000 7fcbd9400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd415dc90 con 0x7fcbd415cdc0 2024-01-21T22:52:40.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.418+0000 7fcbd3e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcbd415cdc0 0x7fcbd40c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:40.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.418+0000 7fcbcbe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fcbd40ca7b0 0x7fcbd406c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:60452/0 (socket says 172.21.15.138:60452) 2024-01-21T22:52:40.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.418+0000 7fcbcbe00640 1 -- 172.21.15.138:0/2254561317 learned_addr learned my addr 172.21.15.138:0/2254561317 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:52:40.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.421+0000 7fcbd3e00640 1 -- 172.21.15.138:0/2254561317 >> v2:172.21.15.138:3300/0 conn(0x7fcbd415d1a0 msgr2=0x7fcbd40c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:40.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.421+0000 7fcbd3e00640 1 --2- 172.21.15.138:0/2254561317 >> v2:172.21.15.138:3300/0 conn(0x7fcbd415d1a0 0x7fcbd40c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:40.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.421+0000 7fcbd3e00640 1 -- 172.21.15.138:0/2254561317 >> v1:172.21.15.152:6789/0 conn(0x7fcbd40ca7b0 legacy=0x7fcbd406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:40.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.421+0000 7fcbd3e00640 1 -- 172.21.15.138:0/2254561317 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd406b4b0 con 0x7fcbd415cdc0 2024-01-21T22:52:40.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.423+0000 7fcbd3e00640 1 --2- 172.21.15.138:0/2254561317 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcbd415cdc0 0x7fcbd40c9ad0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40097a0 tx=0x7fcbc402b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41ace39ef4753147 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:40.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.427+0000 7fcbd3400640 1 -- 172.21.15.138:0/2254561317 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4030610 con 0x7fcbd415cdc0 2024-01-21T22:52:40.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.427+0000 7fcbd3400640 1 -- 172.21.15.138:0/2254561317 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc4030770 con 0x7fcbd415cdc0 2024-01-21T22:52:40.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.427+0000 7fcbd3400640 1 -- 172.21.15.138:0/2254561317 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4030b00 con 0x7fcbd415cdc0 2024-01-21T22:52:40.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.427+0000 7fcbd9400640 1 -- 172.21.15.138:0/2254561317 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcbd415cdc0 msgr2=0x7fcbd40c9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:40.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.427+0000 7fcbd9400640 1 --2- 172.21.15.138:0/2254561317 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcbd415cdc0 0x7fcbd40c9ad0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40097a0 tx=0x7fcbc402b960 comp rx=0 tx=0).stop 2024-01-21T22:52:40.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.428+0000 7fcbd9400640 1 -- 172.21.15.138:0/2254561317 shutdown_connections 2024-01-21T22:52:40.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.428+0000 7fcbd9400640 1 --2- 172.21.15.138:0/2254561317 >> v2:172.21.15.138:3300/0 conn(0x7fcbd415d1a0 0x7fcbd40c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:40.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.428+0000 7fcbd9400640 1 --2- 172.21.15.138:0/2254561317 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcbd415cdc0 0x7fcbd40c9ad0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:40.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.428+0000 7fcbd9400640 1 -- 172.21.15.138:0/2254561317 >> 172.21.15.138:0/2254561317 conn(0x7fcbd406d370 msgr2=0x7fcbd4066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:40.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.428+0000 7fcbd9400640 1 -- 172.21.15.138:0/2254561317 shutdown_connections 2024-01-21T22:52:40.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.428+0000 7fcbd9400640 1 -- 172.21.15.138:0/2254561317 wait complete. 2024-01-21T22:52:40.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.430+0000 7fcbd9400640 1 Processor -- start 2024-01-21T22:52:40.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.430+0000 7fcbd9400640 1 -- start start 2024-01-21T22:52:40.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.431+0000 7fcbd9400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbd40ca7b0 0x7fcbd40cd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:40.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.431+0000 7fcbd9400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcbd415cdc0 0x7fcbd4149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:40.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.431+0000 7fcbd9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbd415d1a0 0x7fcbd414a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:40.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.431+0000 7fcbd9400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd4157e20 con 0x7fcbd415d1a0 2024-01-21T22:52:40.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.431+0000 7fcbd9400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd40c6800 con 0x7fcbd40ca7b0 2024-01-21T22:52:40.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.431+0000 7fcbd9400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd40c70a0 con 0x7fcbd415cdc0 2024-01-21T22:52:40.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.432+0000 7fcbd3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbd40ca7b0 0x7fcbd40cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:40.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.432+0000 7fcbcbe00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcbd415cdc0 0x7fcbd4149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:40.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.432+0000 7fcbd3e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbd40ca7b0 0x7fcbd40cd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:48174/0 (socket says 172.21.15.138:48174) 2024-01-21T22:52:40.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.432+0000 7fcbd3e00640 1 -- 172.21.15.138:0/3831519335 learned_addr learned my addr 172.21.15.138:0/3831519335 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:40.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.436+0000 7fcbcbe00640 1 -- 172.21.15.138:0/3831519335 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbd40ca7b0 msgr2=0x7fcbd40cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:40.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.436+0000 7fcbcbe00640 1 --2- 172.21.15.138:0/3831519335 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbd40ca7b0 0x7fcbd40cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:40.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.436+0000 7fcbcbe00640 1 -- 172.21.15.138:0/3831519335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbd415d1a0 msgr2=0x7fcbd414a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:40.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.436+0000 7fcbcbe00640 1 --2- 172.21.15.138:0/3831519335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbd415d1a0 0x7fcbd414a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:40.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.436+0000 7fcbcbe00640 1 -- 172.21.15.138:0/3831519335 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc4009400 con 0x7fcbd415cdc0 2024-01-21T22:52:40.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.436+0000 7fcbd3e00640 1 --2- 172.21.15.138:0/3831519335 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbd40ca7b0 0x7fcbd40cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-21T22:52:40.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.438+0000 7fcbcbe00640 1 --2- 172.21.15.138:0/3831519335 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcbd415cdc0 0x7fcbd4149bc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc000f60 tx=0x7fcbbc00c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:40.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.440+0000 7fcbd0c00640 1 -- 172.21.15.138:0/3831519335 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc005ce0 con 0x7fcbd415cdc0 2024-01-21T22:52:40.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.441+0000 7fcbd9400640 1 -- 172.21.15.138:0/3831519335 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd41e5590 con 0x7fcbd415cdc0 2024-01-21T22:52:40.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.441+0000 7fcbd9400640 1 -- 172.21.15.138:0/3831519335 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd414fe90 con 0x7fcbd415cdc0 2024-01-21T22:52:40.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.442+0000 7fcbd9400640 1 -- 172.21.15.138:0/3831519335 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd40c7210 con 0x7fcbd415cdc0 2024-01-21T22:52:40.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.442+0000 7fcbd0c00640 1 -- 172.21.15.138:0/3831519335 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbbc005e40 con 0x7fcbd415cdc0 2024-01-21T22:52:40.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.442+0000 7fcbd0c00640 1 -- 172.21.15.138:0/3831519335 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc0072d0 con 0x7fcbd415cdc0 2024-01-21T22:52:40.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.696+0000 7fcbd0c00640 1 -- 172.21.15.138:0/3831519335 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fcbbc0074f0 con 0x7fcbd415cdc0 2024-01-21T22:52:40.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.698+0000 7fcbd0c00640 1 --2- 172.21.15.138:0/3831519335 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcbac06b330 0x7fcbac06d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:40.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.699+0000 7fcbd3e00640 1 --2- 172.21.15.138:0/3831519335 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcbac06b330 0x7fcbac06d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:40.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.700+0000 7fcbd3e00640 1 --2- 172.21.15.138:0/3831519335 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcbac06b330 0x7fcbac06d7f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcbc402b9a0 tx=0x7fcbc403b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:40.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.712+0000 7fcbd0c00640 1 -- 172.21.15.138:0/3831519335 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fcbbc058190 con 0x7fcbd415cdc0 2024-01-21T22:52:40.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.722+0000 7fcbd0c00640 1 -- 172.21.15.138:0/3831519335 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fcbbc087e60 con 0x7fcbd415cdc0 2024-01-21T22:52:40.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.947+0000 7fcbd9400640 1 -- 172.21.15.138:0/3831519335 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcbd4074ee0 con 0x7fcbd415cdc0 2024-01-21T22:52:40.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.962+0000 7fcbd0c00640 1 -- 172.21.15.138:0/3831519335 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcbbc021020 con 0x7fcbd415cdc0 2024-01-21T22:52:40.964 INFO:teuthology.orchestra.run.smithi138.stderr:listed 0 entries 2024-01-21T22:52:40.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.966+0000 7fcbd9400640 1 -- 172.21.15.138:0/3831519335 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcbac06b330 msgr2=0x7fcbac06d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:40.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.966+0000 7fcbd9400640 1 --2- 172.21.15.138:0/3831519335 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcbac06b330 0x7fcbac06d7f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcbc402b9a0 tx=0x7fcbc403b040 comp rx=0 tx=0).stop 2024-01-21T22:52:40.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.966+0000 7fcbd9400640 1 -- 172.21.15.138:0/3831519335 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcbd415cdc0 msgr2=0x7fcbd4149bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:40.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.966+0000 7fcbd9400640 1 --2- 172.21.15.138:0/3831519335 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcbd415cdc0 0x7fcbd4149bc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc000f60 tx=0x7fcbbc00c760 comp rx=0 tx=0).stop 2024-01-21T22:52:40.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.966+0000 7fcbd9400640 1 -- 172.21.15.138:0/3831519335 shutdown_connections 2024-01-21T22:52:40.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.966+0000 7fcbd9400640 1 --2- 172.21.15.138:0/3831519335 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fcbac06b330 0x7fcbac06d7f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:40.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.966+0000 7fcbd9400640 1 --2- 172.21.15.138:0/3831519335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbd415d1a0 0x7fcbd414a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:40.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.966+0000 7fcbd9400640 1 --2- 172.21.15.138:0/3831519335 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fcbd415cdc0 0x7fcbd4149bc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:40.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.966+0000 7fcbd9400640 1 --2- 172.21.15.138:0/3831519335 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fcbd40ca7b0 0x7fcbd40cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:40.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.966+0000 7fcbd9400640 1 -- 172.21.15.138:0/3831519335 >> 172.21.15.138:0/3831519335 conn(0x7fcbd406d370 msgr2=0x7fcbd40cbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:40.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.966+0000 7fcbd9400640 1 -- 172.21.15.138:0/3831519335 shutdown_connections 2024-01-21T22:52:40.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:40.966+0000 7fcbd9400640 1 -- 172.21.15.138:0/3831519335 wait complete. 2024-01-21T22:52:40.987 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-01-21T22:52:40.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-21T22:52:40.987 DEBUG:teuthology.orchestra.run.smithi138:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-01-21T22:52:41.369 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-01-21T22:52:41.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-21T22:52:41.369 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-01-21T22:52:41.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-21T22:52:41.428 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-01-21T22:52:41.472 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-01-21T22:52:41.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-21T22:52:41.473 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-01-21T22:52:41.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-01-21T22:52:41.584 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-01-21T22:52:41.697 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-01-21T22:52:41.707 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-01-21T22:52:41.707 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-21T22:52:41.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.893+0000 7f7fa0400640 1 Processor -- start 2024-01-21T22:52:41.895 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.893+0000 7f7fa0400640 1 -- start start 2024-01-21T22:52:41.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.895+0000 7f7fa0400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f9815a250 0x7f7f9815a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:41.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.895+0000 7f7fa0400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f7f9806a6f0 0x7f7f9806ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:41.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.895+0000 7f7fa0400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f980c61c0 con 0x7f7f98153cc0 2024-01-21T22:52:41.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.895+0000 7f7fa0400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f98159b20 con 0x7f7f9806a6f0 2024-01-21T22:52:41.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.895+0000 7f7fa0400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f98159dc0 con 0x7f7f9815a250 2024-01-21T22:52:41.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.920+0000 7f7f9d000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f9815a250 0x7f7f9815a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:41.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.921+0000 7f7f9d000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f9815a250 0x7f7f9815a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:55188/0 (socket says 172.21.15.138:55188) 2024-01-21T22:52:41.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.921+0000 7f7f9d000640 1 -- 172.21.15.138:0/2250708210 learned_addr learned my addr 172.21.15.138:0/2250708210 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:41.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.924+0000 7f7f9d000640 1 -- 172.21.15.138:0/2250708210 >> v2:172.21.15.138:3300/0 conn(0x7f7f9806a6f0 msgr2=0x7f7f9806ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:41.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.924+0000 7f7f9d000640 1 --2- 172.21.15.138:0/2250708210 >> v2:172.21.15.138:3300/0 conn(0x7f7f9806a6f0 0x7f7f9806ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:41.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.924+0000 7f7f9d000640 1 -- 172.21.15.138:0/2250708210 >> v1:172.21.15.152:6789/0 conn(0x7f7f98153cc0 legacy=0x7f7f980c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:41.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.924+0000 7f7f9d000640 1 -- 172.21.15.138:0/2250708210 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f98071270 con 0x7f7f9815a250 2024-01-21T22:52:41.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.926+0000 7f7f9d000640 1 --2- 172.21.15.138:0/2250708210 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f9815a250 0x7f7f9815a680 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009870 tx=0x7f7f8002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e597c012052a78d8 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:41.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.930+0000 7f7f97e00640 1 -- 172.21.15.138:0/2250708210 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f80031640 con 0x7f7f9815a250 2024-01-21T22:52:41.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.930+0000 7f7f97e00640 1 -- 172.21.15.138:0/2250708210 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f800317a0 con 0x7f7f9815a250 2024-01-21T22:52:41.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.930+0000 7f7f97e00640 1 -- 172.21.15.138:0/2250708210 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f80031b30 con 0x7f7f9815a250 2024-01-21T22:52:41.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.931+0000 7f7fa0400640 1 -- 172.21.15.138:0/2250708210 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f9815a250 msgr2=0x7f7f9815a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:41.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.931+0000 7f7fa0400640 1 --2- 172.21.15.138:0/2250708210 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f9815a250 0x7f7f9815a680 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7f80009870 tx=0x7f7f8002b8f0 comp rx=0 tx=0).stop 2024-01-21T22:52:41.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.932+0000 7f7fa0400640 1 -- 172.21.15.138:0/2250708210 shutdown_connections 2024-01-21T22:52:41.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.932+0000 7f7fa0400640 1 --2- 172.21.15.138:0/2250708210 >> v2:172.21.15.138:3300/0 conn(0x7f7f9806a6f0 0x7f7f9806ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:41.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.932+0000 7f7fa0400640 1 --2- 172.21.15.138:0/2250708210 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f9815a250 0x7f7f9815a680 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:41.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.932+0000 7f7fa0400640 1 -- 172.21.15.138:0/2250708210 >> 172.21.15.138:0/2250708210 conn(0x7f7f980cb0b0 msgr2=0x7f7f980cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:41.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.932+0000 7f7fa0400640 1 -- 172.21.15.138:0/2250708210 shutdown_connections 2024-01-21T22:52:41.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.932+0000 7f7fa0400640 1 -- 172.21.15.138:0/2250708210 wait complete. 2024-01-21T22:52:41.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.934+0000 7f7fa0400640 1 Processor -- start 2024-01-21T22:52:41.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.934+0000 7f7fa0400640 1 -- start start 2024-01-21T22:52:41.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.936+0000 7f7fa0400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f9806a6f0 0x7f7f98155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:41.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.936+0000 7f7fa0400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f98153cc0 0x7f7f980c4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:41.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.936+0000 7f7fa0400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f9815a250 0x7f7f9815bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:41.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.936+0000 7f7fa0400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9815e0c0 con 0x7f7f98153cc0 2024-01-21T22:52:41.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.936+0000 7f7fa0400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f981f3810 con 0x7f7f9806a6f0 2024-01-21T22:52:41.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.936+0000 7f7fa0400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f981f3c30 con 0x7f7f9815a250 2024-01-21T22:52:41.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.937+0000 7f7f9da00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f9806a6f0 0x7f7f98155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:41.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.937+0000 7f7f9e400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f9815a250 0x7f7f9815bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:41.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.937+0000 7f7f9da00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f9806a6f0 0x7f7f98155b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:48190/0 (socket says 172.21.15.138:48190) 2024-01-21T22:52:41.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.937+0000 7f7f9da00640 1 -- 172.21.15.138:0/1969759073 learned_addr learned my addr 172.21.15.138:0/1969759073 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:41.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.941+0000 7f7f9e400640 1 -- 172.21.15.138:0/1969759073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f9806a6f0 msgr2=0x7f7f98155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:41.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.941+0000 7f7f9e400640 1 --2- 172.21.15.138:0/1969759073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f9806a6f0 0x7f7f98155b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:41.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.941+0000 7f7f9e400640 1 -- 172.21.15.138:0/1969759073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f98153cc0 msgr2=0x7f7f980c4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:41.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.941+0000 7f7f9e400640 1 --2- 172.21.15.138:0/1969759073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f98153cc0 0x7f7f980c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:41.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.941+0000 7f7f9e400640 1 -- 172.21.15.138:0/1969759073 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f800094d0 con 0x7f7f9815a250 2024-01-21T22:52:41.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.941+0000 7f7f9da00640 1 --2- 172.21.15.138:0/1969759073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f9806a6f0 0x7f7f98155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-21T22:52:41.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.943+0000 7f7f9e400640 1 --2- 172.21.15.138:0/1969759073 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f9815a250 0x7f7f9815bf40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000be50 tx=0x7f7f9000bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:41.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.949+0000 7f7f95600640 1 -- 172.21.15.138:0/1969759073 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9000ce90 con 0x7f7f9815a250 2024-01-21T22:52:41.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.949+0000 7f7f95600640 1 -- 172.21.15.138:0/1969759073 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f90014440 con 0x7f7f9815a250 2024-01-21T22:52:41.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.949+0000 7f7f95600640 1 -- 172.21.15.138:0/1969759073 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f900147d0 con 0x7f7f9815a250 2024-01-21T22:52:41.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.949+0000 7f7fa0400640 1 -- 172.21.15.138:0/1969759073 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f9815dde0 con 0x7f7f9815a250 2024-01-21T22:52:41.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.950+0000 7f7fa0400640 1 -- 172.21.15.138:0/1969759073 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f981f3150 con 0x7f7f9815a250 2024-01-21T22:52:41.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.951+0000 7f7fa0400640 1 -- 172.21.15.138:0/1969759073 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f981f3950 con 0x7f7f9815a250 2024-01-21T22:52:41.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.971+0000 7f7f95600640 1 -- 172.21.15.138:0/1969759073 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f7f9001c360 con 0x7f7f9815a250 2024-01-21T22:52:41.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.973+0000 7f7f95600640 1 --2- 172.21.15.138:0/1969759073 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7f7006b380 0x7f7f7006d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:41.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.974+0000 7f7f9da00640 1 --2- 172.21.15.138:0/1969759073 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7f7006b380 0x7f7f7006d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:41.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.974+0000 7f7f9da00640 1 --2- 172.21.15.138:0/1969759073 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7f7006b380 0x7f7f7006d840 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7f88001c80 tx=0x7f7f88009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:41.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:41.976+0000 7f7f95600640 1 -- 172.21.15.138:0/1969759073 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7f90058b70 con 0x7f7f9815a250 2024-01-21T22:52:42.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.211+0000 7f7f95600640 1 -- 172.21.15.138:0/1969759073 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f7f90054eb0 con 0x7f7f9815a250 2024-01-21T22:52:42.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.427+0000 7f7fa0400640 1 -- 172.21.15.138:0/1969759073 --> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7f98071f10 con 0x7f7f7006b380 2024-01-21T22:52:42.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.437+0000 7f7f95600640 1 -- 172.21.15.138:0/1969759073 <== mgr.4111 v2:172.21.15.152:6800/314625564 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289589 (secure 0 0 0) 0x7f7f98071f10 con 0x7f7f7006b380 2024-01-21T22:52:42.443 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:52:42.445 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-01-21T22:52:42.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.444+0000 7f7fa0400640 1 -- 172.21.15.138:0/1969759073 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7f7006b380 msgr2=0x7f7f7006d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:42.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.444+0000 7f7fa0400640 1 --2- 172.21.15.138:0/1969759073 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7f7006b380 0x7f7f7006d840 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7f88001c80 tx=0x7f7f88009040 comp rx=0 tx=0).stop 2024-01-21T22:52:42.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.444+0000 7f7fa0400640 1 -- 172.21.15.138:0/1969759073 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f9815a250 msgr2=0x7f7f9815bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:42.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.444+0000 7f7fa0400640 1 --2- 172.21.15.138:0/1969759073 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f9815a250 0x7f7f9815bf40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000be50 tx=0x7f7f9000bf40 comp rx=0 tx=0).stop 2024-01-21T22:52:42.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.444+0000 7f7fa0400640 1 -- 172.21.15.138:0/1969759073 shutdown_connections 2024-01-21T22:52:42.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.444+0000 7f7fa0400640 1 --2- 172.21.15.138:0/1969759073 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7f7006b380 0x7f7f7006d840 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:42.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.444+0000 7f7fa0400640 1 --2- 172.21.15.138:0/1969759073 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7f9815a250 0x7f7f9815bf40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:42.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.444+0000 7f7fa0400640 1 --2- 172.21.15.138:0/1969759073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f98153cc0 0x7f7f980c4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:42.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.444+0000 7f7fa0400640 1 --2- 172.21.15.138:0/1969759073 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7f9806a6f0 0x7f7f98155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:42.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.444+0000 7f7fa0400640 1 -- 172.21.15.138:0/1969759073 >> 172.21.15.138:0/1969759073 conn(0x7f7f980cb0b0 msgr2=0x7f7f9806abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:42.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.445+0000 7f7fa0400640 1 -- 172.21.15.138:0/1969759073 shutdown_connections 2024-01-21T22:52:42.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.445+0000 7f7fa0400640 1 -- 172.21.15.138:0/1969759073 wait complete. 2024-01-21T22:52:42.464 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":858,"stamp":"2024-01-21T22:52:41.537774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":82787731,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":1693,"num_read_kb":212795,"num_write":26807,"num_write_kb":599374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27620,"ondisk_log_size":27620,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":606176,"kb_used_data":163808,"kb_used_omap":12,"kb_used_meta":442291,"kb_avail":754368544,"statfs":{"total":773094113280,"available":772473389056,"internally_reserved":0,"allocated":167739392,"data_stored":166350990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":452906545},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":272532,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":10,"num_write":87,"num_write_kb":322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002650"},"pg_stats":[{"pgid":"3.3f","version":"25'157","reported_seq":192,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.276848+0000","last_change":"2024-01-21T22:25:04.888366+0000","last_active":"2024-01-21T22:52:19.276848+0000","last_peered":"2024-01-21T22:52:19.276848+0000","last_clean":"2024-01-21T22:52:19.276848+0000","last_became_active":"2024-01-21T22:25:04.888060+0000","last_became_peered":"2024-01-21T22:25:04.888060+0000","last_unstale":"2024-01-21T22:52:19.276848+0000","last_undegraded":"2024-01-21T22:52:19.276848+0000","last_fullsized":"2024-01-21T22:52:19.276848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:32:13.512213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":115,"num_write_kb":6171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"25'166","reported_seq":206,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.991866+0000","last_change":"2024-01-21T22:25:04.843704+0000","last_active":"2024-01-21T22:52:19.991866+0000","last_peered":"2024-01-21T22:52:19.991866+0000","last_clean":"2024-01-21T22:52:19.991866+0000","last_became_active":"2024-01-21T22:25:04.843471+0000","last_became_peered":"2024-01-21T22:25:04.843471+0000","last_unstale":"2024-01-21T22:52:19.991866+0000","last_undegraded":"2024-01-21T22:52:19.991866+0000","last_fullsized":"2024-01-21T22:52:19.991866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:34:24.822768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":129,"num_write_kb":4796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"25'195","reported_seq":234,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.221491+0000","last_change":"2024-01-21T22:25:04.858021+0000","last_active":"2024-01-21T22:52:20.221491+0000","last_peered":"2024-01-21T22:52:20.221491+0000","last_clean":"2024-01-21T22:52:20.221491+0000","last_became_active":"2024-01-21T22:25:04.857685+0000","last_became_peered":"2024-01-21T22:25:04.857685+0000","last_unstale":"2024-01-21T22:52:20.221491+0000","last_undegraded":"2024-01-21T22:52:20.221491+0000","last_fullsized":"2024-01-21T22:52:20.221491+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:29:02.950903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":157,"num_write_kb":6223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"25'172","reported_seq":207,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.280308+0000","last_change":"2024-01-21T22:25:04.858941+0000","last_active":"2024-01-21T22:52:19.280308+0000","last_peered":"2024-01-21T22:52:19.280308+0000","last_clean":"2024-01-21T22:52:19.280308+0000","last_became_active":"2024-01-21T22:25:04.858278+0000","last_became_peered":"2024-01-21T22:25:04.858278+0000","last_unstale":"2024-01-21T22:52:19.280308+0000","last_undegraded":"2024-01-21T22:52:19.280308+0000","last_fullsized":"2024-01-21T22:52:19.280308+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T10:03:50.093635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":135,"num_write_kb":19244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"25'177","reported_seq":214,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.448004+0000","last_change":"2024-01-21T22:25:04.881014+0000","last_active":"2024-01-21T22:52:23.448004+0000","last_peered":"2024-01-21T22:52:23.448004+0000","last_clean":"2024-01-21T22:52:23.448004+0000","last_became_active":"2024-01-21T22:25:04.878457+0000","last_became_peered":"2024-01-21T22:25:04.878457+0000","last_unstale":"2024-01-21T22:52:23.448004+0000","last_undegraded":"2024-01-21T22:52:23.448004+0000","last_fullsized":"2024-01-21T22:52:23.448004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:34:24.822768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":119,"num_write_kb":7041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"25'193","reported_seq":221,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.279603+0000","last_change":"2024-01-21T22:25:04.859776+0000","last_active":"2024-01-21T22:52:19.279603+0000","last_peered":"2024-01-21T22:52:19.279603+0000","last_clean":"2024-01-21T22:52:19.279603+0000","last_became_active":"2024-01-21T22:25:04.858731+0000","last_became_peered":"2024-01-21T22:25:04.858731+0000","last_unstale":"2024-01-21T22:52:19.279603+0000","last_undegraded":"2024-01-21T22:52:19.279603+0000","last_fullsized":"2024-01-21T22:52:19.279603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:01:30.091588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":148,"num_write_kb":2956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"25'207","reported_seq":251,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.135091+0000","last_change":"2024-01-21T22:25:04.881392+0000","last_active":"2024-01-21T22:52:20.135091+0000","last_peered":"2024-01-21T22:52:20.135091+0000","last_clean":"2024-01-21T22:52:20.135091+0000","last_became_active":"2024-01-21T22:25:04.881090+0000","last_became_peered":"2024-01-21T22:25:04.881090+0000","last_unstale":"2024-01-21T22:52:20.135091+0000","last_undegraded":"2024-01-21T22:52:20.135091+0000","last_fullsized":"2024-01-21T22:52:20.135091+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:25:50.380424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":4572,"num_write":149,"num_write_kb":5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"25'165","reported_seq":204,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.138549+0000","last_change":"2024-01-21T22:25:04.879186+0000","last_active":"2024-01-21T22:52:20.138549+0000","last_peered":"2024-01-21T22:52:20.138549+0000","last_clean":"2024-01-21T22:52:20.138549+0000","last_became_active":"2024-01-21T22:25:04.878824+0000","last_became_peered":"2024-01-21T22:25:04.878824+0000","last_unstale":"2024-01-21T22:52:20.138549+0000","last_undegraded":"2024-01-21T22:52:20.138549+0000","last_fullsized":"2024-01-21T22:52:20.138549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:31:25.985811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":137,"num_write_kb":11909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"25'151","reported_seq":184,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.378252+0000","last_change":"2024-01-21T22:25:04.886391+0000","last_active":"2024-01-21T22:52:23.378252+0000","last_peered":"2024-01-21T22:52:23.378252+0000","last_clean":"2024-01-21T22:52:23.378252+0000","last_became_active":"2024-01-21T22:25:04.884979+0000","last_became_peered":"2024-01-21T22:25:04.884979+0000","last_unstale":"2024-01-21T22:52:23.378252+0000","last_undegraded":"2024-01-21T22:52:23.378252+0000","last_fullsized":"2024-01-21T22:52:23.378252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:36:36.009632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":96,"num_write_kb":3924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"25'163","reported_seq":202,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.131949+0000","last_change":"2024-01-21T22:25:04.888126+0000","last_active":"2024-01-21T22:52:20.131949+0000","last_peered":"2024-01-21T22:52:20.131949+0000","last_clean":"2024-01-21T22:52:20.131949+0000","last_became_active":"2024-01-21T22:25:04.887631+0000","last_became_peered":"2024-01-21T22:25:04.887631+0000","last_unstale":"2024-01-21T22:52:20.131949+0000","last_undegraded":"2024-01-21T22:52:20.131949+0000","last_fullsized":"2024-01-21T22:52:20.131949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:00:02.684745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":122,"num_write_kb":9871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"25'186","reported_seq":220,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.221571+0000","last_change":"2024-01-21T22:25:04.885677+0000","last_active":"2024-01-21T22:52:20.221571+0000","last_peered":"2024-01-21T22:52:20.221571+0000","last_clean":"2024-01-21T22:52:20.221571+0000","last_became_active":"2024-01-21T22:25:04.884697+0000","last_became_peered":"2024-01-21T22:25:04.884697+0000","last_unstale":"2024-01-21T22:52:20.221571+0000","last_undegraded":"2024-01-21T22:52:20.221571+0000","last_fullsized":"2024-01-21T22:52:20.221571+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:44:23.749631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":143,"num_write_kb":7172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"25'139","reported_seq":176,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:07.428734+0000","last_change":"2024-01-21T22:25:04.886799+0000","last_active":"2024-01-21T22:52:07.428734+0000","last_peered":"2024-01-21T22:52:07.428734+0000","last_clean":"2024-01-21T22:52:07.428734+0000","last_became_active":"2024-01-21T22:25:04.886387+0000","last_became_peered":"2024-01-21T22:25:04.886387+0000","last_unstale":"2024-01-21T22:52:07.428734+0000","last_undegraded":"2024-01-21T22:52:07.428734+0000","last_fullsized":"2024-01-21T22:52:07.428734+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:42:39.977125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":92,"num_write_kb":3768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"25'159","reported_seq":192,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.564136+0000","last_change":"2024-01-21T22:25:04.888103+0000","last_active":"2024-01-21T22:52:19.564136+0000","last_peered":"2024-01-21T22:52:19.564136+0000","last_clean":"2024-01-21T22:52:19.564136+0000","last_became_active":"2024-01-21T22:25:04.887908+0000","last_became_peered":"2024-01-21T22:25:04.887908+0000","last_unstale":"2024-01-21T22:52:19.564136+0000","last_undegraded":"2024-01-21T22:52:19.564136+0000","last_fullsized":"2024-01-21T22:52:19.564136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:23:52.444277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":122,"num_write_kb":8957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"25'167","reported_seq":206,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.374792+0000","last_change":"2024-01-21T22:25:04.842750+0000","last_active":"2024-01-21T22:52:23.374792+0000","last_peered":"2024-01-21T22:52:23.374792+0000","last_clean":"2024-01-21T22:52:23.374792+0000","last_became_active":"2024-01-21T22:25:04.842512+0000","last_became_peered":"2024-01-21T22:25:04.842512+0000","last_unstale":"2024-01-21T22:52:23.374792+0000","last_undegraded":"2024-01-21T22:52:23.374792+0000","last_fullsized":"2024-01-21T22:52:23.374792+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:40:05.554704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":120,"num_write_kb":6061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"25'163","reported_seq":219,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.138574+0000","last_change":"2024-01-21T22:25:04.876592+0000","last_active":"2024-01-21T22:52:20.138574+0000","last_peered":"2024-01-21T22:52:20.138574+0000","last_clean":"2024-01-21T22:52:20.138574+0000","last_became_active":"2024-01-21T22:25:04.876305+0000","last_became_peered":"2024-01-21T22:25:04.876305+0000","last_unstale":"2024-01-21T22:52:20.138574+0000","last_undegraded":"2024-01-21T22:52:20.138574+0000","last_fullsized":"2024-01-21T22:52:20.138574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:32.177562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":7620,"num_write":119,"num_write_kb":10742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"25'200","reported_seq":246,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.273197+0000","last_change":"2024-01-21T22:25:04.878038+0000","last_active":"2024-01-21T22:52:23.273197+0000","last_peered":"2024-01-21T22:52:23.273197+0000","last_clean":"2024-01-21T22:52:23.273197+0000","last_became_active":"2024-01-21T22:25:04.877716+0000","last_became_peered":"2024-01-21T22:25:04.877716+0000","last_unstale":"2024-01-21T22:52:23.273197+0000","last_undegraded":"2024-01-21T22:52:23.273197+0000","last_fullsized":"2024-01-21T22:52:23.273197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:32:13.512213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":5080,"num_write":159,"num_write_kb":7672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"25'175","reported_seq":212,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.145332+0000","last_change":"2024-01-21T22:25:04.890537+0000","last_active":"2024-01-21T22:52:20.145332+0000","last_peered":"2024-01-21T22:52:20.145332+0000","last_clean":"2024-01-21T22:52:20.145332+0000","last_became_active":"2024-01-21T22:25:04.890298+0000","last_became_peered":"2024-01-21T22:25:04.890298+0000","last_unstale":"2024-01-21T22:52:20.145332+0000","last_undegraded":"2024-01-21T22:52:20.145332+0000","last_fullsized":"2024-01-21T22:52:20.145332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:23:57.435028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":136,"num_write_kb":7248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"25'149","reported_seq":184,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.940326+0000","last_change":"2024-01-21T22:25:04.712868+0000","last_active":"2024-01-21T22:52:19.940326+0000","last_peered":"2024-01-21T22:52:19.940326+0000","last_clean":"2024-01-21T22:52:19.940326+0000","last_became_active":"2024-01-21T22:25:04.712616+0000","last_became_peered":"2024-01-21T22:25:04.712616+0000","last_unstale":"2024-01-21T22:52:19.940326+0000","last_undegraded":"2024-01-21T22:52:19.940326+0000","last_fullsized":"2024-01-21T22:52:19.940326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:50:41.306582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":102,"num_write_kb":4431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"25'156","reported_seq":196,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.470695+0000","last_change":"2024-01-21T22:25:04.879328+0000","last_active":"2024-01-21T22:52:19.470695+0000","last_peered":"2024-01-21T22:52:19.470695+0000","last_clean":"2024-01-21T22:52:19.470695+0000","last_became_active":"2024-01-21T22:25:04.879011+0000","last_became_peered":"2024-01-21T22:25:04.879011+0000","last_unstale":"2024-01-21T22:52:19.470695+0000","last_undegraded":"2024-01-21T22:52:19.470695+0000","last_fullsized":"2024-01-21T22:52:19.470695+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:31:23.569672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":108,"num_write_kb":6073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"25'235","reported_seq":276,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.448463+0000","last_change":"2024-01-21T22:25:04.885620+0000","last_active":"2024-01-21T22:52:23.448463+0000","last_peered":"2024-01-21T22:52:23.448463+0000","last_clean":"2024-01-21T22:52:23.448463+0000","last_became_active":"2024-01-21T22:25:04.884364+0000","last_became_peered":"2024-01-21T22:25:04.884364+0000","last_unstale":"2024-01-21T22:52:23.448463+0000","last_undegraded":"2024-01-21T22:52:23.448463+0000","last_fullsized":"2024-01-21T22:52:23.448463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:13:04.601911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3556,"num_write":191,"num_write_kb":16584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"25'189","reported_seq":228,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.934988+0000","last_change":"2024-01-21T22:25:04.868371+0000","last_active":"2024-01-21T22:52:19.934988+0000","last_peered":"2024-01-21T22:52:19.934988+0000","last_clean":"2024-01-21T22:52:19.934988+0000","last_became_active":"2024-01-21T22:25:04.868196+0000","last_became_peered":"2024-01-21T22:25:04.868196+0000","last_unstale":"2024-01-21T22:52:19.934988+0000","last_undegraded":"2024-01-21T22:52:19.934988+0000","last_fullsized":"2024-01-21T22:52:19.934988+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:05:41.950232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":139,"num_write_kb":3732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"25'193","reported_seq":250,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.563845+0000","last_change":"2024-01-21T22:25:04.879144+0000","last_active":"2024-01-21T22:52:19.563845+0000","last_peered":"2024-01-21T22:52:19.563845+0000","last_clean":"2024-01-21T22:52:19.563845+0000","last_became_active":"2024-01-21T22:25:04.878868+0000","last_became_peered":"2024-01-21T22:25:04.878868+0000","last_unstale":"2024-01-21T22:52:19.563845+0000","last_undegraded":"2024-01-21T22:52:19.563845+0000","last_fullsized":"2024-01-21T22:52:19.563845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:55:28.069411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":7620,"num_write":153,"num_write_kb":6732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"25'165","reported_seq":201,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.934401+0000","last_change":"2024-01-21T22:25:04.715246+0000","last_active":"2024-01-21T22:52:19.934401+0000","last_peered":"2024-01-21T22:52:19.934401+0000","last_clean":"2024-01-21T22:52:19.934401+0000","last_became_active":"2024-01-21T22:25:04.714969+0000","last_became_peered":"2024-01-21T22:25:04.714969+0000","last_unstale":"2024-01-21T22:52:19.934401+0000","last_undegraded":"2024-01-21T22:52:19.934401+0000","last_fullsized":"2024-01-21T22:52:19.934401+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:56:24.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":106,"num_write_kb":4123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"25'176","reported_seq":224,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.406727+0000","last_change":"2024-01-21T22:25:04.877902+0000","last_active":"2024-01-21T22:52:19.406727+0000","last_peered":"2024-01-21T22:52:19.406727+0000","last_clean":"2024-01-21T22:52:19.406727+0000","last_became_active":"2024-01-21T22:25:04.876963+0000","last_became_peered":"2024-01-21T22:25:04.876963+0000","last_unstale":"2024-01-21T22:52:19.406727+0000","last_undegraded":"2024-01-21T22:52:19.406727+0000","last_fullsized":"2024-01-21T22:52:19.406727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:30:48.864454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":5334,"num_write":137,"num_write_kb":8161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"25'182","reported_seq":232,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.412606+0000","last_change":"2024-01-21T22:25:04.716735+0000","last_active":"2024-01-21T22:52:23.412606+0000","last_peered":"2024-01-21T22:52:23.412606+0000","last_clean":"2024-01-21T22:52:23.412606+0000","last_became_active":"2024-01-21T22:25:04.716570+0000","last_became_peered":"2024-01-21T22:25:04.716570+0000","last_unstale":"2024-01-21T22:52:23.412606+0000","last_undegraded":"2024-01-21T22:52:23.412606+0000","last_fullsized":"2024-01-21T22:52:23.412606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:44:55.158990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":5842,"num_write":145,"num_write_kb":5757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"25'161","reported_seq":202,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.395083+0000","last_change":"2024-01-21T22:25:04.886230+0000","last_active":"2024-01-21T22:52:23.395083+0000","last_peered":"2024-01-21T22:52:23.395083+0000","last_clean":"2024-01-21T22:52:23.395083+0000","last_became_active":"2024-01-21T22:25:04.885523+0000","last_became_peered":"2024-01-21T22:25:04.885523+0000","last_unstale":"2024-01-21T22:52:23.395083+0000","last_undegraded":"2024-01-21T22:52:23.395083+0000","last_fullsized":"2024-01-21T22:52:23.395083+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:06:02.934920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3556,"num_write":106,"num_write_kb":5820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"25'207","reported_seq":243,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.561247+0000","last_change":"2024-01-21T22:25:04.842692+0000","last_active":"2024-01-21T22:52:19.561247+0000","last_peered":"2024-01-21T22:52:19.561247+0000","last_clean":"2024-01-21T22:52:19.561247+0000","last_became_active":"2024-01-21T22:25:04.842429+0000","last_became_peered":"2024-01-21T22:25:04.842429+0000","last_unstale":"2024-01-21T22:52:19.561247+0000","last_undegraded":"2024-01-21T22:52:19.561247+0000","last_fullsized":"2024-01-21T22:52:19.561247+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:52:24.233709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":167,"num_write_kb":10319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"25'179","reported_seq":218,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.378526+0000","last_change":"2024-01-21T22:25:04.888296+0000","last_active":"2024-01-21T22:52:23.378526+0000","last_peered":"2024-01-21T22:52:23.378526+0000","last_clean":"2024-01-21T22:52:23.378526+0000","last_became_active":"2024-01-21T22:25:04.888025+0000","last_became_peered":"2024-01-21T22:25:04.888025+0000","last_unstale":"2024-01-21T22:52:23.378526+0000","last_undegraded":"2024-01-21T22:52:23.378526+0000","last_fullsized":"2024-01-21T22:52:23.378526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:57:44.129490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":136,"num_write_kb":6753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"25'192","reported_seq":232,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.222625+0000","last_change":"2024-01-21T22:25:04.711908+0000","last_active":"2024-01-21T22:52:20.222625+0000","last_peered":"2024-01-21T22:52:20.222625+0000","last_clean":"2024-01-21T22:52:20.222625+0000","last_became_active":"2024-01-21T22:25:04.711654+0000","last_became_peered":"2024-01-21T22:25:04.711654+0000","last_unstale":"2024-01-21T22:52:20.222625+0000","last_undegraded":"2024-01-21T22:52:20.222625+0000","last_fullsized":"2024-01-21T22:52:20.222625+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:20:41.612265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":148,"num_write_kb":7328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"25'326","reported_seq":367,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.144991+0000","last_change":"2024-01-21T22:25:04.876757+0000","last_active":"2024-01-21T22:52:20.144991+0000","last_peered":"2024-01-21T22:52:20.144991+0000","last_clean":"2024-01-21T22:52:20.144991+0000","last_became_active":"2024-01-21T22:25:04.876012+0000","last_became_peered":"2024-01-21T22:25:04.876012+0000","last_unstale":"2024-01-21T22:52:20.144991+0000","last_undegraded":"2024-01-21T22:52:20.144991+0000","last_fullsized":"2024-01-21T22:52:20.144991+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:00:56.239773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":275,"num_write_kb":7140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"25'195","reported_seq":231,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.940456+0000","last_change":"2024-01-21T22:25:04.715277+0000","last_active":"2024-01-21T22:52:19.940456+0000","last_peered":"2024-01-21T22:52:19.940456+0000","last_clean":"2024-01-21T22:52:19.940456+0000","last_became_active":"2024-01-21T22:25:04.715034+0000","last_became_peered":"2024-01-21T22:25:04.715034+0000","last_unstale":"2024-01-21T22:52:19.940456+0000","last_undegraded":"2024-01-21T22:52:19.940456+0000","last_fullsized":"2024-01-21T22:52:19.940456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:55:28.069411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":140,"num_write_kb":6049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"25'201","reported_seq":233,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.010555+0000","last_change":"2024-01-21T22:25:04.844472+0000","last_active":"2024-01-21T22:52:20.010555+0000","last_peered":"2024-01-21T22:52:20.010555+0000","last_clean":"2024-01-21T22:52:20.010555+0000","last_became_active":"2024-01-21T22:25:04.843827+0000","last_became_peered":"2024-01-21T22:25:04.843827+0000","last_unstale":"2024-01-21T22:52:20.010555+0000","last_undegraded":"2024-01-21T22:52:20.010555+0000","last_fullsized":"2024-01-21T22:52:20.010555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:58:10.552987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1016,"num_write":149,"num_write_kb":12027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"25'151","reported_seq":188,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.381267+0000","last_change":"2024-01-21T22:25:04.713764+0000","last_active":"2024-01-21T22:52:23.381267+0000","last_peered":"2024-01-21T22:52:23.381267+0000","last_clean":"2024-01-21T22:52:23.381267+0000","last_became_active":"2024-01-21T22:25:04.712814+0000","last_became_peered":"2024-01-21T22:25:04.712814+0000","last_unstale":"2024-01-21T22:52:23.381267+0000","last_undegraded":"2024-01-21T22:52:23.381267+0000","last_fullsized":"2024-01-21T22:52:23.381267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:16:41.440927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":111,"num_write_kb":4009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"25'751","reported_seq":778,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.145057+0000","last_change":"2024-01-21T22:25:03.703079+0000","last_active":"2024-01-21T22:52:33.145057+0000","last_peered":"2024-01-21T22:52:33.145057+0000","last_clean":"2024-01-21T22:52:33.145057+0000","last_became_active":"2024-01-21T22:25:03.702776+0000","last_became_peered":"2024-01-21T22:25:03.702776+0000","last_unstale":"2024-01-21T22:52:33.145057+0000","last_undegraded":"2024-01-21T22:52:33.145057+0000","last_fullsized":"2024-01-21T22:52:33.145057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:34:13.302149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":748,"num_write_kb":4569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"25'190","reported_seq":227,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.558673+0000","last_change":"2024-01-21T22:25:04.877158+0000","last_active":"2024-01-21T22:52:19.558673+0000","last_peered":"2024-01-21T22:52:19.558673+0000","last_clean":"2024-01-21T22:52:19.558673+0000","last_became_active":"2024-01-21T22:25:04.876920+0000","last_became_peered":"2024-01-21T22:25:04.876920+0000","last_unstale":"2024-01-21T22:52:19.558673+0000","last_undegraded":"2024-01-21T22:52:19.558673+0000","last_fullsized":"2024-01-21T22:52:19.558673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:46:55.451173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":152,"num_write_kb":6158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"25'776","reported_seq":803,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.474218+0000","last_change":"2024-01-21T22:25:03.720178+0000","last_active":"2024-01-21T22:52:33.474218+0000","last_peered":"2024-01-21T22:52:33.474218+0000","last_clean":"2024-01-21T22:52:33.474218+0000","last_became_active":"2024-01-21T22:25:03.719909+0000","last_became_peered":"2024-01-21T22:25:03.719909+0000","last_unstale":"2024-01-21T22:52:33.474218+0000","last_undegraded":"2024-01-21T22:52:33.474218+0000","last_fullsized":"2024-01-21T22:52:33.474218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:44:53.910872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":769,"num_write_kb":4580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"25'3","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.462701+0000","last_change":"2024-01-21T22:25:03.723005+0000","last_active":"2024-01-21T22:52:28.462701+0000","last_peered":"2024-01-21T22:52:28.462701+0000","last_clean":"2024-01-21T22:52:28.462701+0000","last_became_active":"2024-01-21T22:25:03.722758+0000","last_became_peered":"2024-01-21T22:25:03.722758+0000","last_unstale":"2024-01-21T22:52:28.462701+0000","last_undegraded":"2024-01-21T22:52:28.462701+0000","last_fullsized":"2024-01-21T22:52:28.462701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:16:40.192808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"25'216","reported_seq":255,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.972006+0000","last_change":"2024-01-21T22:25:04.844030+0000","last_active":"2024-01-21T22:52:19.972006+0000","last_peered":"2024-01-21T22:52:19.972006+0000","last_clean":"2024-01-21T22:52:19.972006+0000","last_became_active":"2024-01-21T22:25:04.843714+0000","last_became_peered":"2024-01-21T22:25:04.843714+0000","last_unstale":"2024-01-21T22:52:19.972006+0000","last_undegraded":"2024-01-21T22:52:19.972006+0000","last_fullsized":"2024-01-21T22:52:19.972006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:25:50.380424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":174,"num_write_kb":5546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"25'1195","reported_seq":1222,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.505239+0000","last_change":"2024-01-21T22:25:03.702767+0000","last_active":"2024-01-21T22:52:23.505239+0000","last_peered":"2024-01-21T22:52:23.505239+0000","last_clean":"2024-01-21T22:52:23.505239+0000","last_became_active":"2024-01-21T22:25:03.702140+0000","last_became_peered":"2024-01-21T22:25:03.702140+0000","last_unstale":"2024-01-21T22:52:23.505239+0000","last_undegraded":"2024-01-21T22:52:23.505239+0000","last_fullsized":"2024-01-21T22:52:23.505239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:46:54.203055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1190,"num_write_kb":8787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"25'188","reported_seq":226,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.121546+0000","last_change":"2024-01-21T22:25:04.867173+0000","last_active":"2024-01-21T22:52:20.121546+0000","last_peered":"2024-01-21T22:52:20.121546+0000","last_clean":"2024-01-21T22:52:20.121546+0000","last_became_active":"2024-01-21T22:25:04.866945+0000","last_became_peered":"2024-01-21T22:25:04.866945+0000","last_unstale":"2024-01-21T22:52:20.121546+0000","last_undegraded":"2024-01-21T22:52:20.121546+0000","last_fullsized":"2024-01-21T22:52:20.121546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:15:41.725132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":136,"num_write_kb":8996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.462303+0000","last_change":"2024-01-21T22:25:03.715128+0000","last_active":"2024-01-21T22:52:23.462303+0000","last_peered":"2024-01-21T22:52:23.462303+0000","last_clean":"2024-01-21T22:52:23.462303+0000","last_became_active":"2024-01-21T22:25:03.714039+0000","last_became_peered":"2024-01-21T22:25:03.714039+0000","last_unstale":"2024-01-21T22:52:23.462303+0000","last_undegraded":"2024-01-21T22:52:23.462303+0000","last_fullsized":"2024-01-21T22:52:23.462303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:09:09.988399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"25'200","reported_seq":238,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.220935+0000","last_change":"2024-01-21T22:25:04.886534+0000","last_active":"2024-01-21T22:52:20.220935+0000","last_peered":"2024-01-21T22:52:20.220935+0000","last_clean":"2024-01-21T22:52:20.220935+0000","last_became_active":"2024-01-21T22:25:04.885963+0000","last_became_peered":"2024-01-21T22:25:04.885963+0000","last_unstale":"2024-01-21T22:52:20.220935+0000","last_undegraded":"2024-01-21T22:52:20.220935+0000","last_fullsized":"2024-01-21T22:52:20.220935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:44:31.867220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":149,"num_write_kb":6741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"25'4","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.470132+0000","last_change":"2024-01-21T22:25:03.718237+0000","last_active":"2024-01-21T22:52:28.470132+0000","last_peered":"2024-01-21T22:52:28.470132+0000","last_clean":"2024-01-21T22:52:28.470132+0000","last_became_active":"2024-01-21T22:25:03.717937+0000","last_became_peered":"2024-01-21T22:25:03.717937+0000","last_unstale":"2024-01-21T22:52:28.470132+0000","last_undegraded":"2024-01-21T22:52:28.470132+0000","last_fullsized":"2024-01-21T22:52:28.470132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:31:24.737692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"25'186","reported_seq":222,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.137695+0000","last_change":"2024-01-21T22:25:04.882743+0000","last_active":"2024-01-21T22:52:20.137695+0000","last_peered":"2024-01-21T22:52:20.137695+0000","last_clean":"2024-01-21T22:52:20.137695+0000","last_became_active":"2024-01-21T22:25:04.882478+0000","last_became_peered":"2024-01-21T22:25:04.882478+0000","last_unstale":"2024-01-21T22:52:20.137695+0000","last_undegraded":"2024-01-21T22:52:20.137695+0000","last_fullsized":"2024-01-21T22:52:20.137695+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:42:39.977125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2540,"num_write":148,"num_write_kb":5415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"25'467","reported_seq":494,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.492525+0000","last_change":"2024-01-21T22:25:03.703672+0000","last_active":"2024-01-21T22:52:33.492525+0000","last_peered":"2024-01-21T22:52:33.492525+0000","last_clean":"2024-01-21T22:52:33.492525+0000","last_became_active":"2024-01-21T22:25:03.703426+0000","last_became_peered":"2024-01-21T22:25:03.703426+0000","last_unstale":"2024-01-21T22:52:33.492525+0000","last_undegraded":"2024-01-21T22:52:33.492525+0000","last_fullsized":"2024-01-21T22:52:33.492525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:41:51.793206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":466,"num_write_kb":463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"25'204","reported_seq":252,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.603931+0000","last_change":"2024-01-21T22:25:04.887306+0000","last_active":"2024-01-21T22:52:19.603931+0000","last_peered":"2024-01-21T22:52:19.603931+0000","last_clean":"2024-01-21T22:52:19.603931+0000","last_became_active":"2024-01-21T22:25:04.886970+0000","last_became_peered":"2024-01-21T22:25:04.886970+0000","last_unstale":"2024-01-21T22:52:19.603931+0000","last_undegraded":"2024-01-21T22:52:19.603931+0000","last_fullsized":"2024-01-21T22:52:19.603931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:59:45.239426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":5588,"num_write":173,"num_write_kb":7144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"25'878","reported_seq":905,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.282057+0000","last_change":"2024-01-21T22:25:03.714102+0000","last_active":"2024-01-21T22:52:19.282057+0000","last_peered":"2024-01-21T22:52:19.282057+0000","last_clean":"2024-01-21T22:52:19.282057+0000","last_became_active":"2024-01-21T22:25:03.713717+0000","last_became_peered":"2024-01-21T22:25:03.713717+0000","last_unstale":"2024-01-21T22:52:19.282057+0000","last_undegraded":"2024-01-21T22:52:19.282057+0000","last_fullsized":"2024-01-21T22:52:19.282057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:32:15.181238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":877,"num_write_kb":4705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"25'163","reported_seq":207,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.933099+0000","last_change":"2024-01-21T22:25:04.881307+0000","last_active":"2024-01-21T22:52:19.933099+0000","last_peered":"2024-01-21T22:52:19.933099+0000","last_clean":"2024-01-21T22:52:19.933099+0000","last_became_active":"2024-01-21T22:25:04.880472+0000","last_became_peered":"2024-01-21T22:25:04.880472+0000","last_unstale":"2024-01-21T22:52:19.933099+0000","last_undegraded":"2024-01-21T22:52:19.933099+0000","last_fullsized":"2024-01-21T22:52:19.933099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:29:02.950903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":4572,"num_write":122,"num_write_kb":6796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"25'1290","reported_seq":1317,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.495049+0000","last_change":"2024-01-21T22:25:03.721825+0000","last_active":"2024-01-21T22:52:33.495049+0000","last_peered":"2024-01-21T22:52:33.495049+0000","last_clean":"2024-01-21T22:52:33.495049+0000","last_became_active":"2024-01-21T22:25:03.721502+0000","last_became_peered":"2024-01-21T22:25:03.721502+0000","last_unstale":"2024-01-21T22:52:33.495049+0000","last_undegraded":"2024-01-21T22:52:33.495049+0000","last_fullsized":"2024-01-21T22:52:33.495049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1290,"log_dups_size":0,"ondisk_log_size":1290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:09:09.988399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1286,"num_write_kb":8849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"25'155","reported_seq":186,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.935419+0000","last_change":"2024-01-21T22:25:04.877455+0000","last_active":"2024-01-21T22:52:19.935419+0000","last_peered":"2024-01-21T22:52:19.935419+0000","last_clean":"2024-01-21T22:52:19.935419+0000","last_became_active":"2024-01-21T22:25:04.877257+0000","last_became_peered":"2024-01-21T22:25:04.877257+0000","last_unstale":"2024-01-21T22:52:19.935419+0000","last_undegraded":"2024-01-21T22:52:19.935419+0000","last_fullsized":"2024-01-21T22:52:19.935419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:34:14.550267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1270,"num_write":117,"num_write_kb":4465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"25'4","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.497453+0000","last_change":"2024-01-21T22:25:03.706077+0000","last_active":"2024-01-21T22:52:28.497453+0000","last_peered":"2024-01-21T22:52:28.497453+0000","last_clean":"2024-01-21T22:52:28.497453+0000","last_became_active":"2024-01-21T22:25:03.702798+0000","last_became_peered":"2024-01-21T22:25:03.702798+0000","last_unstale":"2024-01-21T22:52:28.497453+0000","last_undegraded":"2024-01-21T22:52:28.497453+0000","last_fullsized":"2024-01-21T22:52:28.497453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:20:40.364147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"25'216","reported_seq":251,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.108670+0000","last_change":"2024-01-21T22:25:04.890636+0000","last_active":"2024-01-21T22:52:19.108670+0000","last_peered":"2024-01-21T22:52:19.108670+0000","last_clean":"2024-01-21T22:52:19.108670+0000","last_became_active":"2024-01-21T22:25:04.890271+0000","last_became_peered":"2024-01-21T22:25:04.890271+0000","last_unstale":"2024-01-21T22:52:19.108670+0000","last_undegraded":"2024-01-21T22:52:19.108670+0000","last_fullsized":"2024-01-21T22:52:19.108670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:15:46.754395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":174,"num_write_kb":31551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"25'4","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.470849+0000","last_change":"2024-01-21T22:25:03.717202+0000","last_active":"2024-01-21T22:52:28.470849+0000","last_peered":"2024-01-21T22:52:28.470849+0000","last_clean":"2024-01-21T22:52:28.470849+0000","last_became_active":"2024-01-21T22:25:03.716846+0000","last_became_peered":"2024-01-21T22:25:03.716846+0000","last_unstale":"2024-01-21T22:52:28.470849+0000","last_undegraded":"2024-01-21T22:52:28.470849+0000","last_fullsized":"2024-01-21T22:52:28.470849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:10.904923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"25'163","reported_seq":196,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:07.183268+0000","last_change":"2024-01-21T22:25:04.715053+0000","last_active":"2024-01-21T22:52:07.183268+0000","last_peered":"2024-01-21T22:52:07.183268+0000","last_clean":"2024-01-21T22:52:07.183268+0000","last_became_active":"2024-01-21T22:25:04.713566+0000","last_became_peered":"2024-01-21T22:25:04.713566+0000","last_unstale":"2024-01-21T22:52:07.183268+0000","last_undegraded":"2024-01-21T22:52:07.183268+0000","last_fullsized":"2024-01-21T22:52:07.183268+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:50:41.306582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":119,"num_write_kb":3558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"25'9","reported_seq":36,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:50:43.264719+0000","last_change":"2024-01-21T22:25:03.703017+0000","last_active":"2024-01-21T22:50:43.264719+0000","last_peered":"2024-01-21T22:50:43.264719+0000","last_clean":"2024-01-21T22:50:43.264719+0000","last_became_active":"2024-01-21T22:25:03.702742+0000","last_became_peered":"2024-01-21T22:25:03.702742+0000","last_unstale":"2024-01-21T22:50:43.264719+0000","last_undegraded":"2024-01-21T22:50:43.264719+0000","last_fullsized":"2024-01-21T22:50:43.264719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:42:03.386594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"25'190","reported_seq":223,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.394334+0000","last_change":"2024-01-21T22:25:04.876507+0000","last_active":"2024-01-21T22:52:23.394334+0000","last_peered":"2024-01-21T22:52:23.394334+0000","last_clean":"2024-01-21T22:52:23.394334+0000","last_became_active":"2024-01-21T22:25:04.876169+0000","last_became_peered":"2024-01-21T22:25:04.876169+0000","last_unstale":"2024-01-21T22:52:23.394334+0000","last_undegraded":"2024-01-21T22:52:23.394334+0000","last_fullsized":"2024-01-21T22:52:23.394334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:12:11.958115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":142,"num_write_kb":8431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"25'7","reported_seq":35,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.467670+0000","last_change":"2024-01-21T22:25:03.720257+0000","last_active":"2024-01-21T22:52:28.467670+0000","last_peered":"2024-01-21T22:52:28.467670+0000","last_clean":"2024-01-21T22:52:28.467670+0000","last_became_active":"2024-01-21T22:25:03.720009+0000","last_became_peered":"2024-01-21T22:25:03.720009+0000","last_unstale":"2024-01-21T22:52:28.467670+0000","last_undegraded":"2024-01-21T22:52:28.467670+0000","last_fullsized":"2024-01-21T22:52:28.467670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:37:56.569677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"25'148","reported_seq":196,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.221225+0000","last_change":"2024-01-21T22:25:04.883112+0000","last_active":"2024-01-21T22:52:20.221225+0000","last_peered":"2024-01-21T22:52:20.221225+0000","last_clean":"2024-01-21T22:52:20.221225+0000","last_became_active":"2024-01-21T22:25:04.882767+0000","last_became_peered":"2024-01-21T22:25:04.882767+0000","last_unstale":"2024-01-21T22:52:20.221225+0000","last_undegraded":"2024-01-21T22:52:20.221225+0000","last_fullsized":"2024-01-21T22:52:20.221225+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:18:03.439738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":5334,"num_write":115,"num_write_kb":10137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"25'300","reported_seq":328,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.539211+0000","last_change":"2024-01-21T22:25:03.703516+0000","last_active":"2024-01-21T22:52:33.539211+0000","last_peered":"2024-01-21T22:52:33.539211+0000","last_clean":"2024-01-21T22:52:33.539211+0000","last_became_active":"2024-01-21T22:25:03.703256+0000","last_became_peered":"2024-01-21T22:25:03.703256+0000","last_unstale":"2024-01-21T22:52:33.539211+0000","last_undegraded":"2024-01-21T22:52:33.539211+0000","last_fullsized":"2024-01-21T22:52:33.539211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:57:42.881371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":296,"num_write_kb":4241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"25'157","reported_seq":190,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.449441+0000","last_change":"2024-01-21T22:25:38.276745+0000","last_active":"2024-01-21T22:52:23.449441+0000","last_peered":"2024-01-21T22:52:23.449441+0000","last_clean":"2024-01-21T22:52:23.449441+0000","last_became_active":"2024-01-21T22:25:38.276512+0000","last_became_peered":"2024-01-21T22:25:38.276512+0000","last_unstale":"2024-01-21T22:52:23.449441+0000","last_undegraded":"2024-01-21T22:52:23.449441+0000","last_fullsized":"2024-01-21T22:52:23.449441+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:12.153042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":119,"num_write_kb":11355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"25'627","reported_seq":655,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:03.142512+0000","last_change":"2024-01-21T22:25:03.708366+0000","last_active":"2024-01-21T22:52:03.142512+0000","last_peered":"2024-01-21T22:52:03.142512+0000","last_clean":"2024-01-21T22:52:03.142512+0000","last_became_active":"2024-01-21T22:25:03.708101+0000","last_became_peered":"2024-01-21T22:25:03.708101+0000","last_unstale":"2024-01-21T22:52:03.142512+0000","last_undegraded":"2024-01-21T22:52:03.142512+0000","last_fullsized":"2024-01-21T22:52:03.142512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:57:42.881371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":625,"num_write_kb":4435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"25'192","reported_seq":226,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.937318+0000","last_change":"2024-01-21T22:25:04.877993+0000","last_active":"2024-01-21T22:52:19.937318+0000","last_peered":"2024-01-21T22:52:19.937318+0000","last_clean":"2024-01-21T22:52:19.937318+0000","last_became_active":"2024-01-21T22:25:04.877661+0000","last_became_peered":"2024-01-21T22:25:04.877661+0000","last_unstale":"2024-01-21T22:52:19.937318+0000","last_undegraded":"2024-01-21T22:52:19.937318+0000","last_fullsized":"2024-01-21T22:52:19.937318+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:20:40.947205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":145,"num_write_kb":7036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"25'2","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.462672+0000","last_change":"2024-01-21T22:25:03.721559+0000","last_active":"2024-01-21T22:52:28.462672+0000","last_peered":"2024-01-21T22:52:28.462672+0000","last_clean":"2024-01-21T22:52:28.462672+0000","last_became_active":"2024-01-21T22:25:03.721281+0000","last_became_peered":"2024-01-21T22:25:03.721281+0000","last_unstale":"2024-01-21T22:52:28.462672+0000","last_undegraded":"2024-01-21T22:52:28.462672+0000","last_fullsized":"2024-01-21T22:52:28.462672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:55:19.679388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"25'166","reported_seq":210,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.255338+0000","last_change":"2024-01-21T22:25:04.883087+0000","last_active":"2024-01-21T22:52:19.255338+0000","last_peered":"2024-01-21T22:52:19.255338+0000","last_clean":"2024-01-21T22:52:19.255338+0000","last_became_active":"2024-01-21T22:25:04.880227+0000","last_became_peered":"2024-01-21T22:25:04.880227+0000","last_unstale":"2024-01-21T22:52:19.255338+0000","last_undegraded":"2024-01-21T22:52:19.255338+0000","last_fullsized":"2024-01-21T22:52:19.255338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:58:09.653538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":4318,"num_write":118,"num_write_kb":4353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":181,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:48:53.349412+0000","last_change":"2024-01-21T22:24:52.968324+0000","last_active":"2024-01-21T22:48:53.349412+0000","last_peered":"2024-01-21T22:48:53.349412+0000","last_clean":"2024-01-21T22:48:53.349412+0000","last_became_active":"2024-01-21T22:24:52.967018+0000","last_became_peered":"2024-01-21T22:24:52.967018+0000","last_unstale":"2024-01-21T22:48:53.349412+0000","last_undegraded":"2024-01-21T22:48:53.349412+0000","last_fullsized":"2024-01-21T22:48:53.349412+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:24:50.312540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:24:50.312540+0000","last_clean_scrub_stamp":"2024-01-21T22:24:50.312540+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:52:04.800073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"25'4","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.461845+0000","last_change":"2024-01-21T22:25:03.707407+0000","last_active":"2024-01-21T22:52:33.461845+0000","last_peered":"2024-01-21T22:52:33.461845+0000","last_clean":"2024-01-21T22:52:33.461845+0000","last_became_active":"2024-01-21T22:25:03.707085+0000","last_became_peered":"2024-01-21T22:25:03.707085+0000","last_unstale":"2024-01-21T22:52:33.461845+0000","last_undegraded":"2024-01-21T22:52:33.461845+0000","last_fullsized":"2024-01-21T22:52:33.461845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:44:22.501512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"25'169","reported_seq":207,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.558341+0000","last_change":"2024-01-21T22:25:04.868564+0000","last_active":"2024-01-21T22:52:19.558341+0000","last_peered":"2024-01-21T22:52:19.558341+0000","last_clean":"2024-01-21T22:52:19.558341+0000","last_became_active":"2024-01-21T22:25:04.868247+0000","last_became_peered":"2024-01-21T22:25:04.868247+0000","last_unstale":"2024-01-21T22:52:19.558341+0000","last_undegraded":"2024-01-21T22:52:19.558341+0000","last_fullsized":"2024-01-21T22:52:19.558341+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:30:50.039501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2540,"num_write":132,"num_write_kb":4885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"25'1292","reported_seq":1321,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.530564+0000","last_change":"2024-01-21T22:25:03.706070+0000","last_active":"2024-01-21T22:52:33.530564+0000","last_peered":"2024-01-21T22:52:33.530564+0000","last_clean":"2024-01-21T22:52:33.530564+0000","last_became_active":"2024-01-21T22:25:03.705785+0000","last_became_peered":"2024-01-21T22:25:03.705785+0000","last_unstale":"2024-01-21T22:52:33.530564+0000","last_undegraded":"2024-01-21T22:52:33.530564+0000","last_fullsized":"2024-01-21T22:52:33.530564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1292,"log_dups_size":0,"ondisk_log_size":1292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:30.929444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1286,"num_write_kb":8869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"25'183","reported_seq":223,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.224605+0000","last_change":"2024-01-21T22:25:04.845436+0000","last_active":"2024-01-21T22:52:20.224605+0000","last_peered":"2024-01-21T22:52:20.224605+0000","last_clean":"2024-01-21T22:52:20.224605+0000","last_became_active":"2024-01-21T22:25:04.845274+0000","last_became_peered":"2024-01-21T22:25:04.845274+0000","last_unstale":"2024-01-21T22:52:20.224605+0000","last_undegraded":"2024-01-21T22:52:20.224605+0000","last_fullsized":"2024-01-21T22:52:20.224605+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:08:36.686787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":151,"num_write_kb":10622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"25'754","reported_seq":1527,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.492860+0000","last_change":"2024-01-21T22:25:03.709174+0000","last_active":"2024-01-21T22:52:33.492860+0000","last_peered":"2024-01-21T22:52:33.492860+0000","last_clean":"2024-01-21T22:52:33.492860+0000","last_became_active":"2024-01-21T22:25:03.708894+0000","last_became_peered":"2024-01-21T22:25:03.708894+0000","last_unstale":"2024-01-21T22:52:33.492860+0000","last_undegraded":"2024-01-21T22:52:33.492860+0000","last_fullsized":"2024-01-21T22:52:33.492860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:19:26.591140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":608727,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":744,"num_read_kb":1113,"num_write":748,"num_write_kb":1116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"25'180","reported_seq":222,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.222007+0000","last_change":"2024-01-21T22:25:04.711947+0000","last_active":"2024-01-21T22:52:20.222007+0000","last_peered":"2024-01-21T22:52:20.222007+0000","last_clean":"2024-01-21T22:52:20.222007+0000","last_became_active":"2024-01-21T22:25:04.711694+0000","last_became_peered":"2024-01-21T22:25:04.711694+0000","last_unstale":"2024-01-21T22:52:20.222007+0000","last_undegraded":"2024-01-21T22:52:20.222007+0000","last_fullsized":"2024-01-21T22:52:20.222007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:37:57.817795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3556,"num_write":135,"num_write_kb":5944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"25'1294","reported_seq":1323,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:08.470169+0000","last_change":"2024-01-21T22:25:03.706355+0000","last_active":"2024-01-21T22:52:08.470169+0000","last_peered":"2024-01-21T22:52:08.470169+0000","last_clean":"2024-01-21T22:52:08.470169+0000","last_became_active":"2024-01-21T22:25:03.703876+0000","last_became_peered":"2024-01-21T22:25:03.703876+0000","last_unstale":"2024-01-21T22:52:08.470169+0000","last_undegraded":"2024-01-21T22:52:08.470169+0000","last_fullsized":"2024-01-21T22:52:08.470169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1294,"log_dups_size":0,"ondisk_log_size":1294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:42:38.729006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1291,"num_write_kb":8882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"25'223","reported_seq":260,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.222078+0000","last_change":"2024-01-21T22:25:04.885997+0000","last_active":"2024-01-21T22:52:20.222078+0000","last_peered":"2024-01-21T22:52:20.222078+0000","last_clean":"2024-01-21T22:52:20.222078+0000","last_became_active":"2024-01-21T22:25:04.884162+0000","last_became_peered":"2024-01-21T22:25:04.884162+0000","last_unstale":"2024-01-21T22:52:20.222078+0000","last_undegraded":"2024-01-21T22:52:20.222078+0000","last_fullsized":"2024-01-21T22:52:20.222078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:44:55.158990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":180,"num_write_kb":15842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"25'3","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.542219+0000","last_change":"2024-01-21T22:25:03.708300+0000","last_active":"2024-01-21T22:52:33.542219+0000","last_peered":"2024-01-21T22:52:33.542219+0000","last_clean":"2024-01-21T22:52:33.542219+0000","last_became_active":"2024-01-21T22:25:03.708036+0000","last_became_peered":"2024-01-21T22:25:03.708036+0000","last_unstale":"2024-01-21T22:52:33.542219+0000","last_undegraded":"2024-01-21T22:52:33.542219+0000","last_fullsized":"2024-01-21T22:52:33.542219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:23:51.196159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"25'184","reported_seq":228,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.941920+0000","last_change":"2024-01-21T22:25:04.844713+0000","last_active":"2024-01-21T22:52:19.941920+0000","last_peered":"2024-01-21T22:52:19.941920+0000","last_clean":"2024-01-21T22:52:19.941920+0000","last_became_active":"2024-01-21T22:25:04.844008+0000","last_became_peered":"2024-01-21T22:25:04.844008+0000","last_unstale":"2024-01-21T22:52:19.941920+0000","last_undegraded":"2024-01-21T22:52:19.941920+0000","last_fullsized":"2024-01-21T22:52:19.941920+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:33:25.760064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":4064,"num_write":142,"num_write_kb":4003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"25'2","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.495347+0000","last_change":"2024-01-21T22:25:03.699093+0000","last_active":"2024-01-21T22:52:28.495347+0000","last_peered":"2024-01-21T22:52:28.495347+0000","last_clean":"2024-01-21T22:52:28.495347+0000","last_became_active":"2024-01-21T22:25:03.698443+0000","last_became_peered":"2024-01-21T22:25:03.698443+0000","last_unstale":"2024-01-21T22:52:28.495347+0000","last_undegraded":"2024-01-21T22:52:28.495347+0000","last_fullsized":"2024-01-21T22:52:28.495347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:20:39.699087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"25'204","reported_seq":239,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.126809+0000","last_change":"2024-01-21T22:25:04.842733+0000","last_active":"2024-01-21T22:52:20.126809+0000","last_peered":"2024-01-21T22:52:20.126809+0000","last_clean":"2024-01-21T22:52:20.126809+0000","last_became_active":"2024-01-21T22:25:04.842479+0000","last_became_peered":"2024-01-21T22:25:04.842479+0000","last_unstale":"2024-01-21T22:52:20.126809+0000","last_undegraded":"2024-01-21T22:52:20.126809+0000","last_fullsized":"2024-01-21T22:52:20.126809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:51:06.931790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":156,"num_write_kb":5205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"25'1417","reported_seq":1445,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:36.159123+0000","last_change":"2024-01-21T22:25:03.720306+0000","last_active":"2024-01-21T22:52:36.159123+0000","last_peered":"2024-01-21T22:52:36.159123+0000","last_clean":"2024-01-21T22:52:36.159123+0000","last_became_active":"2024-01-21T22:25:03.720067+0000","last_became_peered":"2024-01-21T22:25:03.720067+0000","last_unstale":"2024-01-21T22:52:36.159123+0000","last_undegraded":"2024-01-21T22:52:36.159123+0000","last_fullsized":"2024-01-21T22:52:36.159123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1417,"log_dups_size":0,"ondisk_log_size":1417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:52:22.985590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3483,"num_write_kb":1511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"25'220","reported_seq":270,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.252712+0000","last_change":"2024-01-21T22:25:04.859628+0000","last_active":"2024-01-21T22:52:20.252712+0000","last_peered":"2024-01-21T22:52:20.252712+0000","last_clean":"2024-01-21T22:52:20.252712+0000","last_became_active":"2024-01-21T22:25:04.859270+0000","last_became_peered":"2024-01-21T22:25:04.859270+0000","last_unstale":"2024-01-21T22:52:20.252712+0000","last_undegraded":"2024-01-21T22:52:20.252712+0000","last_fullsized":"2024-01-21T22:52:20.252712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:56:24.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":5842,"num_write":177,"num_write_kb":7531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"25'6","reported_seq":34,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.531715+0000","last_change":"2024-01-21T22:25:03.722006+0000","last_active":"2024-01-21T22:52:33.531715+0000","last_peered":"2024-01-21T22:52:33.531715+0000","last_clean":"2024-01-21T22:52:33.531715+0000","last_became_active":"2024-01-21T22:25:03.721275+0000","last_became_peered":"2024-01-21T22:25:03.721275+0000","last_unstale":"2024-01-21T22:52:33.531715+0000","last_undegraded":"2024-01-21T22:52:33.531715+0000","last_fullsized":"2024-01-21T22:52:33.531715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:43:53.801587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"25'175","reported_seq":215,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.145841+0000","last_change":"2024-01-21T22:25:04.858864+0000","last_active":"2024-01-21T22:52:20.145841+0000","last_peered":"2024-01-21T22:52:20.145841+0000","last_clean":"2024-01-21T22:52:20.145841+0000","last_became_active":"2024-01-21T22:25:04.858586+0000","last_became_peered":"2024-01-21T22:25:04.858586+0000","last_unstale":"2024-01-21T22:52:20.145841+0000","last_undegraded":"2024-01-21T22:52:20.145841+0000","last_fullsized":"2024-01-21T22:52:20.145841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:32.177562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":132,"num_write_kb":7745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"25'670","reported_seq":698,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.532991+0000","last_change":"2024-01-21T22:25:03.718436+0000","last_active":"2024-01-21T22:52:33.532991+0000","last_peered":"2024-01-21T22:52:33.532991+0000","last_clean":"2024-01-21T22:52:33.532991+0000","last_became_active":"2024-01-21T22:25:03.718171+0000","last_became_peered":"2024-01-21T22:25:03.718171+0000","last_unstale":"2024-01-21T22:52:33.532991+0000","last_undegraded":"2024-01-21T22:52:33.532991+0000","last_fullsized":"2024-01-21T22:52:33.532991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:05:40.702113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":666,"num_write_kb":4450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"25'191","reported_seq":227,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:07.603453+0000","last_change":"2024-01-21T22:25:04.875736+0000","last_active":"2024-01-21T22:52:07.603453+0000","last_peered":"2024-01-21T22:52:07.603453+0000","last_clean":"2024-01-21T22:52:07.603453+0000","last_became_active":"2024-01-21T22:25:04.875368+0000","last_became_peered":"2024-01-21T22:25:04.875368+0000","last_unstale":"2024-01-21T22:52:07.603453+0000","last_undegraded":"2024-01-21T22:52:07.603453+0000","last_fullsized":"2024-01-21T22:52:07.603453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:39:31.273250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":138,"num_write_kb":6957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.491094+0000","last_change":"2024-01-21T22:25:03.722820+0000","last_active":"2024-01-21T22:52:28.491094+0000","last_peered":"2024-01-21T22:52:28.491094+0000","last_clean":"2024-01-21T22:52:28.491094+0000","last_became_active":"2024-01-21T22:25:03.722525+0000","last_became_peered":"2024-01-21T22:25:03.722525+0000","last_unstale":"2024-01-21T22:52:28.491094+0000","last_undegraded":"2024-01-21T22:52:28.491094+0000","last_fullsized":"2024-01-21T22:52:28.491094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:40:04.306585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"25'169","reported_seq":209,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.378058+0000","last_change":"2024-01-21T22:25:04.857451+0000","last_active":"2024-01-21T22:52:23.378058+0000","last_peered":"2024-01-21T22:52:23.378058+0000","last_clean":"2024-01-21T22:52:23.378058+0000","last_became_active":"2024-01-21T22:25:04.857205+0000","last_became_peered":"2024-01-21T22:25:04.857205+0000","last_unstale":"2024-01-21T22:52:23.378058+0000","last_undegraded":"2024-01-21T22:52:23.378058+0000","last_fullsized":"2024-01-21T22:52:23.378058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:12.153042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":126,"num_write_kb":4840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"25'2","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:51:08.155593+0000","last_change":"2024-01-21T22:25:03.708574+0000","last_active":"2024-01-21T22:51:08.155593+0000","last_peered":"2024-01-21T22:51:08.155593+0000","last_clean":"2024-01-21T22:51:08.155593+0000","last_became_active":"2024-01-21T22:25:03.708318+0000","last_became_peered":"2024-01-21T22:25:03.708318+0000","last_unstale":"2024-01-21T22:51:08.155593+0000","last_undegraded":"2024-01-21T22:51:08.155593+0000","last_fullsized":"2024-01-21T22:51:08.155593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:50:40.058463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"25'214","reported_seq":274,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.129443+0000","last_change":"2024-01-21T22:25:04.843079+0000","last_active":"2024-01-21T22:52:20.129443+0000","last_peered":"2024-01-21T22:52:20.129443+0000","last_clean":"2024-01-21T22:52:20.129443+0000","last_became_active":"2024-01-21T22:25:04.842815+0000","last_became_peered":"2024-01-21T22:25:04.842815+0000","last_unstale":"2024-01-21T22:52:20.129443+0000","last_undegraded":"2024-01-21T22:52:20.129443+0000","last_fullsized":"2024-01-21T22:52:20.129443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:16:41.440927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":8382,"num_write":174,"num_write_kb":9170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"25'763","reported_seq":791,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.532623+0000","last_change":"2024-01-21T22:25:03.710506+0000","last_active":"2024-01-21T22:52:33.532623+0000","last_peered":"2024-01-21T22:52:33.532623+0000","last_clean":"2024-01-21T22:52:33.532623+0000","last_became_active":"2024-01-21T22:25:03.710262+0000","last_became_peered":"2024-01-21T22:25:03.710262+0000","last_unstale":"2024-01-21T22:52:33.532623+0000","last_undegraded":"2024-01-21T22:52:33.532623+0000","last_fullsized":"2024-01-21T22:52:33.532623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:25:49.132306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":756,"num_write_kb":4574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"25'188","reported_seq":229,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.941642+0000","last_change":"2024-01-21T22:25:04.876469+0000","last_active":"2024-01-21T22:52:19.941642+0000","last_peered":"2024-01-21T22:52:19.941642+0000","last_clean":"2024-01-21T22:52:19.941642+0000","last_became_active":"2024-01-21T22:25:04.875619+0000","last_became_peered":"2024-01-21T22:25:04.875619+0000","last_unstale":"2024-01-21T22:52:19.941642+0000","last_undegraded":"2024-01-21T22:52:19.941642+0000","last_fullsized":"2024-01-21T22:52:19.941642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:30:20.167607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3556,"num_write":140,"num_write_kb":4746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"25'140","reported_seq":173,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.144836+0000","last_change":"2024-01-21T22:25:04.891067+0000","last_active":"2024-01-21T22:52:20.144836+0000","last_peered":"2024-01-21T22:52:20.144836+0000","last_clean":"2024-01-21T22:52:20.144836+0000","last_became_active":"2024-01-21T22:25:04.890515+0000","last_became_peered":"2024-01-21T22:25:04.890515+0000","last_unstale":"2024-01-21T22:52:20.144836+0000","last_undegraded":"2024-01-21T22:52:20.144836+0000","last_fullsized":"2024-01-21T22:52:20.144836+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:25:48.726591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":88,"num_write_kb":3797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"25'196","reported_seq":223,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:35.717345+0000","last_change":"2024-01-21T22:25:03.719412+0000","last_active":"2024-01-21T22:52:35.717345+0000","last_peered":"2024-01-21T22:52:35.717345+0000","last_clean":"2024-01-21T22:52:35.717345+0000","last_became_active":"2024-01-21T22:25:03.718163+0000","last_became_peered":"2024-01-21T22:25:03.718163+0000","last_unstale":"2024-01-21T22:52:35.717345+0000","last_undegraded":"2024-01-21T22:52:35.717345+0000","last_fullsized":"2024-01-21T22:52:35.717345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:56:22.817420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1895390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":195,"num_write_kb":1969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"25'203","reported_seq":235,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.379097+0000","last_change":"2024-01-21T22:25:04.889166+0000","last_active":"2024-01-21T22:52:23.379097+0000","last_peered":"2024-01-21T22:52:23.379097+0000","last_clean":"2024-01-21T22:52:23.379097+0000","last_became_active":"2024-01-21T22:25:04.888910+0000","last_became_peered":"2024-01-21T22:25:04.888910+0000","last_unstale":"2024-01-21T22:52:23.379097+0000","last_undegraded":"2024-01-21T22:52:23.379097+0000","last_fullsized":"2024-01-21T22:52:23.379097+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:55:53.892439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":155,"num_write_kb":13062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"25'1484","reported_seq":1511,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.469678+0000","last_change":"2024-01-21T22:25:03.718932+0000","last_active":"2024-01-21T22:52:33.469678+0000","last_peered":"2024-01-21T22:52:33.469678+0000","last_clean":"2024-01-21T22:52:33.469678+0000","last_became_active":"2024-01-21T22:25:03.718649+0000","last_became_peered":"2024-01-21T22:25:03.718649+0000","last_unstale":"2024-01-21T22:52:33.469678+0000","last_undegraded":"2024-01-21T22:52:33.469678+0000","last_fullsized":"2024-01-21T22:52:33.469678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1484,"log_dups_size":0,"ondisk_log_size":1484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:30.929444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1478,"num_write_kb":9119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"25'204","reported_seq":259,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.264524+0000","last_change":"2024-01-21T22:25:04.715429+0000","last_active":"2024-01-21T22:52:19.264524+0000","last_peered":"2024-01-21T22:52:19.264524+0000","last_clean":"2024-01-21T22:52:19.264524+0000","last_became_active":"2024-01-21T22:25:04.714720+0000","last_became_peered":"2024-01-21T22:25:04.714720+0000","last_unstale":"2024-01-21T22:52:19.264524+0000","last_undegraded":"2024-01-21T22:52:19.264524+0000","last_fullsized":"2024-01-21T22:52:19.264524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:44:23.749631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":6858,"num_write":147,"num_write_kb":17332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"25'747","reported_seq":776,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.457894+0000","last_change":"2024-01-21T22:25:03.703289+0000","last_active":"2024-01-21T22:52:28.457894+0000","last_peered":"2024-01-21T22:52:28.457894+0000","last_clean":"2024-01-21T22:52:28.457894+0000","last_became_active":"2024-01-21T22:25:03.703020+0000","last_became_peered":"2024-01-21T22:25:03.703020+0000","last_unstale":"2024-01-21T22:52:28.457894+0000","last_undegraded":"2024-01-21T22:52:28.457894+0000","last_fullsized":"2024-01-21T22:52:28.457894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:10.904923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":743,"num_write_kb":4567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"25'211","reported_seq":261,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.392685+0000","last_change":"2024-01-21T22:25:04.715894+0000","last_active":"2024-01-21T22:52:23.392685+0000","last_peered":"2024-01-21T22:52:23.392685+0000","last_clean":"2024-01-21T22:52:23.392685+0000","last_became_active":"2024-01-21T22:25:04.715652+0000","last_became_peered":"2024-01-21T22:25:04.715652+0000","last_unstale":"2024-01-21T22:52:23.392685+0000","last_undegraded":"2024-01-21T22:52:23.392685+0000","last_fullsized":"2024-01-21T22:52:23.392685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:13:04.601911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":5588,"num_write":182,"num_write_kb":13243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"25'762","reported_seq":791,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.498304+0000","last_change":"2024-01-21T22:25:03.702339+0000","last_active":"2024-01-21T22:52:23.498304+0000","last_peered":"2024-01-21T22:52:23.498304+0000","last_clean":"2024-01-21T22:52:23.498304+0000","last_became_active":"2024-01-21T22:25:03.702087+0000","last_became_peered":"2024-01-21T22:25:03.702087+0000","last_unstale":"2024-01-21T22:52:23.498304+0000","last_undegraded":"2024-01-21T22:52:23.498304+0000","last_fullsized":"2024-01-21T22:52:23.498304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":762,"log_dups_size":0,"ondisk_log_size":762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:51:05.683671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":760,"num_write_kb":4560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":833,"num_read_kb":211582,"num_write":8969,"num_write_kb":506711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11784,"ondisk_log_size":11784,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":82197363,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":744,"num_read_kb":1113,"num_write":17679,"num_write_kb":90247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164454400,"data_stored":164394726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15719,"ondisk_log_size":15719,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542483,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62372,"kb_used_data":13148,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94309468,"statfs":{"total":96636764160,"available":96572895232,"internally_reserved":0,"allocated":13463552,"data_stored":13288704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67700000000000005,"15min":0.68500000000000005},"min":{"1min":0.46000000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89600000000000002,"5min":1.0189999999999999,"15min":1.0640000000000001},"last":0.496},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70699999999999996,"15min":0.69399999999999995},"min":{"1min":0.497,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.081,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.376}]},{"osd":1,"last update":"Sun Jan 21 22:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71599999999999997,"15min":0.71099999999999997},"min":{"1min":0.55000000000000004,"5min":0.51700000000000002,"15min":0.34000000000000002},"max":{"1min":0.96899999999999997,"5min":1.159,"15min":1.4179999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.73399999999999999,"15min":0.72299999999999998},"min":{"1min":0.56000000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95599999999999996,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.42999999999999999}]},{"osd":2,"last update":"Sun Jan 21 22:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69299999999999995,"15min":0.70099999999999996},"min":{"1min":0.49399999999999999,"5min":0.46000000000000002,"15min":0.39300000000000002},"max":{"1min":0.94699999999999995,"5min":1.0069999999999999,"15min":1.052},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73899999999999999,"15min":0.72499999999999998},"min":{"1min":0.58099999999999996,"5min":0.438,"15min":0.38},"max":{"1min":1.02,"5min":1.1140000000000001,"15min":1.1799999999999999},"last":0.55000000000000004}]},{"osd":3,"last update":"Sun Jan 21 22:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71099999999999997,"15min":0.71399999999999997},"min":{"1min":0.54500000000000004,"5min":0.42699999999999999,"15min":0.42099999999999999},"max":{"1min":1.048,"5min":1.093,"15min":1.1539999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.73199999999999998,"15min":0.74099999999999999},"min":{"1min":0.56299999999999994,"5min":0.48399999999999999,"15min":0.45600000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.181},"last":0.46100000000000002}]},{"osd":4,"last update":"Sun Jan 21 22:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.35099999999999998,"15min":0.34499999999999997},"min":{"1min":0.26600000000000001,"5min":0.245,"15min":0.22700000000000001},"max":{"1min":0.39800000000000002,"5min":0.57099999999999995,"15min":0.58299999999999996},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.35199999999999998,"15min":0.34699999999999998},"min":{"1min":0.27300000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.41699999999999998,"5min":0.66700000000000004,"15min":0.68100000000000005},"last":0.44800000000000001}]},{"osd":5,"last update":"Sun Jan 21 22:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34300000000000003,"15min":0.34200000000000003},"min":{"1min":0.255,"5min":0.255,"15min":0.249},"max":{"1min":0.44500000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38100000000000001,"15min":0.372},"min":{"1min":0.31,"5min":0.26700000000000002,"15min":0.249},"max":{"1min":0.48599999999999999,"5min":0.54100000000000004,"15min":0.624},"last":0.39200000000000002}]},{"osd":6,"last update":"Sun Jan 21 22:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.40000000000000002,"15min":0.39500000000000002},"min":{"1min":0.33400000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.45200000000000001,"5min":0.57199999999999995,"15min":0.67300000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.313,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.52300000000000002,"5min":0.60699999999999998,"15min":0.63700000000000001},"last":0.41099999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542483,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43720,"kb_used_data":12544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328120,"statfs":{"total":96636764160,"available":96591994880,"internally_reserved":0,"allocated":12845056,"data_stored":12679853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69299999999999995,"15min":0.70099999999999996},"min":{"1min":0.497,"5min":0.48299999999999998,"15min":0.45600000000000002},"max":{"1min":0.89200000000000002,"5min":1.0189999999999999,"15min":3.3380000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.54100000000000004,"5min":0.495,"15min":0.44600000000000001},"max":{"1min":0.93799999999999994,"5min":1.038,"15min":1.0669999999999999},"last":0.92300000000000004}]},{"osd":1,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72099999999999997,"15min":0.70699999999999996},"min":{"1min":0.46500000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0529999999999999,"5min":1.0860000000000001,"15min":2.9860000000000002},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72299999999999998,"15min":0.72099999999999997},"min":{"1min":0.52900000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.1020000000000001},"last":0.57999999999999996}]},{"osd":2,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.49299999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.88200000000000001,"5min":1.1220000000000001,"15min":7.5439999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71999999999999997,"15min":0.73199999999999998},"min":{"1min":0.59499999999999997,"5min":0.56200000000000006,"15min":0.495},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.81399999999999995}]},{"osd":3,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72399999999999998,"15min":0.70499999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.42599999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73099999999999998,"15min":0.78100000000000003},"min":{"1min":0.56399999999999995,"5min":0.52700000000000002,"15min":0.39400000000000002},"max":{"1min":0.90100000000000002,"5min":1.075,"15min":8.4689999999999994},"last":0.73199999999999998}]},{"osd":4,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34300000000000003,"15min":0.33800000000000002},"min":{"1min":0.26000000000000001,"5min":0.245,"15min":0.222},"max":{"1min":0.45200000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.35599999999999998,"15min":0.35199999999999998},"min":{"1min":0.24199999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.45100000000000001,"5min":0.45600000000000002,"15min":0.91900000000000004},"last":0.32200000000000001}]},{"osd":5,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35199999999999998,"15min":0.35399999999999998},"min":{"1min":0.30199999999999999,"5min":0.26400000000000001,"15min":0.26100000000000001},"max":{"1min":0.40200000000000002,"5min":0.53800000000000003,"15min":0.61299999999999999},"last":0.374},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.377,"15min":0.36799999999999999},"min":{"1min":0.29599999999999999,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.45300000000000001,"5min":0.46800000000000003,"15min":0.52200000000000002},"last":0.30199999999999999}]},{"osd":7,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.38},"min":{"1min":0.29799999999999999,"5min":0.27100000000000002,"15min":0.26600000000000001},"max":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.96099999999999997},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.39200000000000002,"15min":0.39100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28799999999999998,"15min":0.27300000000000002},"max":{"1min":0.45200000000000001,"5min":0.54900000000000004,"15min":0.98699999999999999},"last":0.33800000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542483,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":101608,"kb_used_data":26016,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94270232,"statfs":{"total":96636764160,"available":96532717568,"internally_reserved":0,"allocated":26640384,"data_stored":26461952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.68400000000000005,"15min":0.68899999999999995},"min":{"1min":0.46300000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.499,"5min":0.432,"15min":0.41499999999999998},"max":{"1min":0.83799999999999997,"5min":1.0149999999999999,"15min":1.1020000000000001},"last":0.55100000000000005}]},{"osd":1,"last update":"Sun Jan 21 22:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67200000000000004,"15min":0.68999999999999995},"min":{"1min":0.56599999999999995,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91500000000000004,"5min":0.95599999999999996,"15min":1.1950000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.74099999999999999,"15min":0.72799999999999998},"min":{"1min":0.50800000000000001,"5min":0.49199999999999999,"15min":0.40999999999999998},"max":{"1min":0.96999999999999997,"5min":1.111,"15min":1.151},"last":0.83699999999999997}]},{"osd":2,"last update":"Sun Jan 21 22:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.72099999999999997,"15min":0.71899999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.434},"max":{"1min":1.046,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70399999999999996,"15min":0.71499999999999997},"min":{"1min":0.53300000000000003,"5min":0.437,"15min":0.437},"max":{"1min":0.85399999999999998,"5min":1.143,"15min":1.149},"last":0.70499999999999996}]},{"osd":3,"last update":"Sun Jan 21 22:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.70799999999999996,"15min":0.73399999999999999},"min":{"1min":0.48499999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97499999999999998,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.71799999999999997},"min":{"1min":0.58799999999999997,"5min":0.46200000000000002,"15min":0.373},"max":{"1min":0.97199999999999998,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.81999999999999995}]},{"osd":4,"last update":"Sun Jan 21 22:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.32800000000000001,"15min":0.33400000000000002},"min":{"1min":0.254,"5min":0.23599999999999999,"15min":0.22800000000000001},"max":{"1min":0.433,"5min":0.46300000000000002,"15min":0.47099999999999997},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.34399999999999997,"15min":0.34999999999999998},"min":{"1min":0.28100000000000003,"5min":0.26500000000000001,"15min":0.254},"max":{"1min":0.41199999999999998,"5min":0.50700000000000001,"15min":0.55400000000000005},"last":0.317}]},{"osd":6,"last update":"Sun Jan 21 22:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.27700000000000002,"5min":0.26300000000000001,"15min":0.26000000000000001},"max":{"1min":0.38600000000000001,"5min":0.48899999999999999,"15min":0.51800000000000002},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.25600000000000001},"max":{"1min":0.41699999999999998,"5min":0.48099999999999998,"15min":0.51100000000000001},"last":0.34999999999999998}]},{"osd":7,"last update":"Sun Jan 21 22:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.38,"15min":0.38200000000000001},"min":{"1min":0.28499999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.441,"5min":0.47799999999999998,"15min":0.64000000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.38200000000000001,"15min":0.38400000000000001},"min":{"1min":0.28199999999999997,"5min":0.27600000000000002,"15min":0.27100000000000002},"max":{"1min":0.45300000000000001,"5min":0.54100000000000004,"15min":0.57099999999999995},"last":0.41399999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542483,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65944,"kb_used_data":12560,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94305896,"statfs":{"total":96636764160,"available":96569237504,"internally_reserved":0,"allocated":12861440,"data_stored":12681197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.51100000000000001,"5min":0.46200000000000002,"15min":0.38800000000000001},"max":{"1min":0.97299999999999998,"5min":1.05,"15min":1.0880000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69399999999999995,"15min":0.71499999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.41099999999999998},"max":{"1min":0.998,"5min":1.008,"15min":2.9649999999999999},"last":0.93400000000000005}]},{"osd":1,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.70599999999999996,"15min":0.71899999999999997},"min":{"1min":0.45800000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.9410000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.70199999999999996,"15min":0.71399999999999997},"min":{"1min":0.44,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.615},"last":0.67100000000000004}]},{"osd":2,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70499999999999996,"15min":0.71499999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.95799999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73499999999999999,"15min":0.71799999999999997},"min":{"1min":0.54700000000000004,"5min":0.505,"15min":0.40400000000000003},"max":{"1min":1.05,"5min":1.05,"15min":1.4710000000000001},"last":0.70099999999999996}]},{"osd":3,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69899999999999995,"15min":0.69799999999999995},"min":{"1min":0.55400000000000005,"5min":0.48499999999999999,"15min":0.43099999999999999},"max":{"1min":0.97499999999999998,"5min":1.0640000000000001,"15min":1.0720000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.72699999999999998,"15min":0.745},"min":{"1min":0.53800000000000003,"5min":0.52300000000000002,"15min":0.50900000000000001},"max":{"1min":0.92700000000000005,"5min":1.0600000000000001,"15min":1.1160000000000001},"last":0.68600000000000005}]},{"osd":5,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34499999999999997,"15min":0.34200000000000003},"min":{"1min":0.251,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.432,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.35099999999999998,"15min":0.34799999999999998},"min":{"1min":0.28000000000000003,"5min":0.252,"15min":0.249},"max":{"1min":0.42299999999999999,"5min":0.443,"15min":0.49399999999999999},"last":0.45300000000000001}]},{"osd":6,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34999999999999998,"15min":0.34699999999999998},"min":{"1min":0.28100000000000003,"5min":0.24299999999999999,"15min":0.23599999999999999},"max":{"1min":0.42099999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.376,"5min":0.378,"15min":0.374},"min":{"1min":0.29199999999999998,"5min":0.27700000000000002,"15min":0.26400000000000001},"max":{"1min":0.46800000000000003,"5min":0.51500000000000001,"15min":0.54000000000000004},"last":0.43099999999999999}]},{"osd":7,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.27600000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.52000000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39600000000000002,"15min":0.39000000000000001},"min":{"1min":0.317,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.47699999999999998,"5min":0.626,"15min":0.626},"last":0.52900000000000003}]}]},{"osd":3,"up_from":14,"seq":60129542484,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104396,"kb_used_data":37124,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94267444,"statfs":{"total":96636764160,"available":96529862656,"internally_reserved":0,"allocated":38014976,"data_stored":37846259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:51:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.069,"15min":1.089},"min":{"1min":0.73999999999999999,"5min":0.69699999999999995,"15min":0.56499999999999995},"max":{"1min":1.8169999999999999,"5min":3.9849999999999999,"15min":6.1879999999999997},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.274,"15min":1.2769999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.67300000000000004},"max":{"1min":1.53,"5min":8.6639999999999997,"15min":8.6639999999999997},"last":1.04}]},{"osd":1,"last update":"Sun Jan 21 22:51:40 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.2889999999999999,"15min":1.2709999999999999},"min":{"1min":0.751,"5min":0.72999999999999998,"15min":0.67000000000000004},"max":{"1min":1.7010000000000001,"5min":17.492999999999999,"15min":17.492999999999999},"last":1.401},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.4379999999999999,"15min":1.423},"min":{"1min":1.05,"5min":1.0269999999999999,"15min":0.874},"max":{"1min":1.635,"5min":8.9629999999999992,"15min":8.9629999999999992},"last":1.1850000000000001}]},{"osd":2,"last update":"Sun Jan 21 22:51:40 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.401,"15min":1.3740000000000001},"min":{"1min":1.115,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.585,"5min":8.5530000000000008,"15min":8.5530000000000008},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.462,"5min":1.5680000000000001,"15min":1.5169999999999999},"min":{"1min":1.3240000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.639,"5min":17.664999999999999,"15min":17.664999999999999},"last":1.304}]},{"osd":4,"last update":"Sun Jan 21 22:51:40 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.169,"15min":1.151},"min":{"1min":0.82299999999999995,"5min":0.78000000000000003,"15min":0.72099999999999997},"max":{"1min":1.5580000000000001,"5min":4.5099999999999998,"15min":4.9080000000000004},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.054,"5min":1.1659999999999999,"15min":1.151},"min":{"1min":0.83499999999999996,"5min":0.77800000000000002,"15min":0.71799999999999997},"max":{"1min":1.304,"5min":5.6559999999999997,"15min":5.6559999999999997},"last":0.81100000000000005}]},{"osd":5,"last update":"Sun Jan 21 22:51:40 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.173,"15min":1.1950000000000001},"min":{"1min":0.93600000000000005,"5min":0.77000000000000002,"15min":0.754},"max":{"1min":1.472,"5min":3.2040000000000002,"15min":9.2370000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.353,"15min":1.3100000000000001},"min":{"1min":0.871,"5min":0.80600000000000005,"15min":0.72399999999999998},"max":{"1min":1.9330000000000001,"5min":8.7420000000000009,"15min":8.7420000000000009},"last":1.089}]},{"osd":6,"last update":"Sun Jan 21 22:51:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3420000000000001,"15min":1.3600000000000001},"min":{"1min":0.88200000000000001,"5min":0.874,"15min":0.85899999999999999},"max":{"1min":1.6839999999999999,"5min":3.3199999999999998,"15min":9.3719999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.3939999999999999,"15min":1.379},"min":{"1min":0.95899999999999996,"5min":0.89300000000000002,"15min":0.78700000000000003},"max":{"1min":1.7949999999999999,"5min":5.806,"15min":5.806},"last":1.224}]},{"osd":7,"last update":"Sun Jan 21 22:51:40 2024","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.476,"15min":1.492},"min":{"1min":1.109,"5min":0.93700000000000006,"15min":0.86099999999999999},"max":{"1min":1.9019999999999999,"5min":3.27,"15min":9.4610000000000003},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.526,"5min":1.5629999999999999,"15min":1.522},"min":{"1min":1.1060000000000001,"5min":1.0580000000000001,"15min":0.97399999999999998},"max":{"1min":2.0339999999999998,"5min":8.8719999999999999,"15min":8.8719999999999999},"last":1.266}]}]},{"osd":2,"up_from":14,"seq":60129542482,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61776,"kb_used_data":12552,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94310064,"statfs":{"total":96636764160,"available":96573505536,"internally_reserved":0,"allocated":12853248,"data_stored":12680525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0509999999999999,"15min":1.04},"min":{"1min":0.79100000000000004,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5369999999999999,"5min":1.9410000000000001,"15min":2.0419999999999998},"last":1.294},{"interface":"front","average":{"1min":1.349,"5min":1.306,"15min":1.2769999999999999},"min":{"1min":0.95599999999999996,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.6299999999999999,"5min":2.2770000000000001,"15min":2.2770000000000001},"last":1.484}]},{"osd":1,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.29,"15min":1.2629999999999999},"min":{"1min":0.88400000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.605,"5min":1.9399999999999999,"15min":2.234},"last":1.431},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.3540000000000001,"15min":1.3759999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.8540000000000001,"5min":2.411,"15min":2.411},"last":1.149}]},{"osd":3,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.27,"15min":1.296},"min":{"1min":0.96799999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.556,"5min":1.788,"15min":1.9690000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.429,"15min":1.4410000000000001},"min":{"1min":1.2230000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.681,"5min":2.0659999999999998,"15min":2.1320000000000001},"last":1.54}]},{"osd":4,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.149,"15min":1.155},"min":{"1min":0.90100000000000002,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.345,"5min":1.776,"15min":2.0259999999999998},"last":1.615},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0720000000000001,"15min":1.0820000000000001},"min":{"1min":0.69999999999999996,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4199999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.88600000000000001}]},{"osd":5,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.79900000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.415,"5min":1.623,"15min":1.669},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.244,"5min":1.2809999999999999,"15min":1.29},"min":{"1min":1.0860000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5720000000000001,"5min":1.8859999999999999,"15min":2.125},"last":1.6639999999999999}]},{"osd":6,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.244,"15min":1.25},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.6160000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.2130000000000001,"15min":1.254},"min":{"1min":0.96799999999999997,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.5880000000000001,"5min":1.8600000000000001,"15min":1.8919999999999999},"last":1.0249999999999999}]},{"osd":7,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.377,"15min":1.4019999999999999},"min":{"1min":1.034,"5min":0.748,"15min":0.748},"max":{"1min":1.7450000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3640000000000001,"15min":1.391},"min":{"1min":1.077,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.694,"5min":1.9139999999999999,"15min":2.004},"last":1.2350000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542483,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76556,"kb_used_data":23172,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94295284,"statfs":{"total":96636764160,"available":96558370816,"internally_reserved":0,"allocated":23728128,"data_stored":23554289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.0620000000000001,"15min":1.0509999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.3839999999999999,"5min":2.3839999999999999,"15min":3.613},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.2430000000000001,"15min":1.214},"min":{"1min":0.76000000000000001,"5min":0.72499999999999998,"15min":0.63800000000000001},"max":{"1min":3.3879999999999999,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.94999999999999996}]},{"osd":2,"last update":"Sun Jan 21 22:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.248,"15min":1.236},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.73799999999999999},"max":{"1min":3.056,"5min":3.056,"15min":4.2309999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.355,"15min":1.353},"min":{"1min":0.996,"5min":0.82499999999999996,"15min":0.76700000000000002},"max":{"1min":2.8159999999999998,"5min":2.8159999999999998,"15min":3.2450000000000001},"last":0.85899999999999999}]},{"osd":3,"last update":"Sun Jan 21 22:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.4339999999999999,"15min":1.425},"min":{"1min":1.0189999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":4.0890000000000004,"5min":4.0890000000000004,"15min":4.0890000000000004},"last":1.218},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.3540000000000001,"15min":1.3560000000000001},"min":{"1min":0.874,"5min":0.84599999999999997,"15min":0.78100000000000003},"max":{"1min":3.8519999999999999,"5min":3.8519999999999999,"15min":3.8519999999999999},"last":1.2689999999999999}]},{"osd":4,"last update":"Sun Jan 21 22:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0660000000000001,"15min":1.028},"min":{"1min":0.67700000000000005,"5min":0.67200000000000004,"15min":0.65600000000000003},"max":{"1min":2.8639999999999999,"5min":2.8639999999999999,"15min":2.8639999999999999},"last":1.006},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.206,"15min":1.1879999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.74299999999999999},"max":{"1min":2.5939999999999999,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":1.3300000000000001}]},{"osd":5,"last update":"Sun Jan 21 22:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1140000000000001,"15min":1.1100000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71099999999999997},"max":{"1min":3.0139999999999998,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":1.143},{"interface":"front","average":{"1min":1.194,"5min":1.26,"15min":1.2410000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.972,"5min":2.972,"15min":2.972},"last":1.099}]},{"osd":6,"last update":"Sun Jan 21 22:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.302,"15min":1.3009999999999999},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.79300000000000004},"max":{"1min":3.121,"5min":3.121,"15min":3.121},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.4279999999999999,"15min":1.399},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.88200000000000001},"max":{"1min":3.3410000000000002,"5min":3.3410000000000002,"15min":3.3410000000000002},"last":1.1899999999999999}]},{"osd":7,"last update":"Sun Jan 21 22:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.86799999999999999},"max":{"1min":2.718,"5min":2.718,"15min":2.718},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4410000000000001,"15min":1.4710000000000001},"min":{"1min":1.002,"5min":1.002,"15min":0.93200000000000005},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":1.4770000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542484,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89804,"kb_used_data":26692,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94282036,"statfs":{"total":96636764160,"available":96544804864,"internally_reserved":0,"allocated":27332608,"data_stored":27158211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0309999999999999,"15min":1.0640000000000001},"min":{"1min":0.70499999999999996,"5min":0.624,"15min":0.621},"max":{"1min":1.6080000000000001,"5min":1.6870000000000001,"15min":1.988},"last":1.208},{"interface":"front","average":{"1min":1.288,"5min":1.232,"15min":1.2889999999999999},"min":{"1min":0.90100000000000002,"5min":0.70199999999999996,"15min":0.67200000000000004},"max":{"1min":1.609,"5min":1.718,"15min":11.01},"last":0.89600000000000002}]},{"osd":2,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.306,"15min":1.3300000000000001},"min":{"1min":1.2050000000000001,"5min":0.83799999999999997,"15min":0.81699999999999995},"max":{"1min":1.8580000000000001,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.3480000000000001,"15min":1.3879999999999999},"min":{"1min":0.96199999999999997,"5min":0.84499999999999997,"15min":0.84399999999999997},"max":{"1min":1.742,"5min":1.9690000000000001,"15min":11.308999999999999},"last":1.127}]},{"osd":3,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.268,"15min":1.3129999999999999},"min":{"1min":0.98999999999999999,"5min":0.85399999999999998,"15min":0.67000000000000004},"max":{"1min":1.573,"5min":1.671,"15min":9.2929999999999993},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.462,"5min":1.4390000000000001,"15min":1.4850000000000001},"min":{"1min":1.1679999999999999,"5min":1.01,"15min":0.91700000000000004},"max":{"1min":1.696,"5min":2.0390000000000001,"15min":11.4},"last":1.302}]},{"osd":4,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0660000000000001,"15min":1.0860000000000001},"min":{"1min":0.82099999999999995,"5min":0.70999999999999996,"15min":0.66200000000000003},"max":{"1min":1.3200000000000001,"5min":1.593,"15min":2.0110000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.157,"15min":1.2050000000000001},"min":{"1min":0.94099999999999995,"5min":0.83899999999999997,"15min":0.77300000000000002},"max":{"1min":1.823,"5min":1.823,"15min":11.345000000000001},"last":1.0029999999999999}]},{"osd":5,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.0289999999999999,"15min":1.0549999999999999},"min":{"1min":0.76200000000000001,"5min":0.71699999999999997,"15min":0.70299999999999996},"max":{"1min":1.498,"5min":1.7470000000000001,"15min":2.0030000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2190000000000001,"15min":1.266},"min":{"1min":0.84399999999999997,"5min":0.82799999999999996,"15min":0.66300000000000003},"max":{"1min":1.4099999999999999,"5min":1.6539999999999999,"15min":2.1509999999999998},"last":0.93300000000000005}]},{"osd":6,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1819999999999999,"15min":1.2130000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.369,"5min":1.871,"15min":2.198},"last":1.04},{"interface":"front","average":{"1min":1.353,"5min":1.3540000000000001,"15min":1.397},"min":{"1min":1.0269999999999999,"5min":0.91000000000000003,"15min":0.872},"max":{"1min":1.74,"5min":1.8120000000000001,"15min":2.2509999999999999},"last":1.0820000000000001}]},{"osd":7,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.48,"5min":1.3680000000000001,"15min":1.4219999999999999},"min":{"1min":1.1839999999999999,"5min":0.96299999999999997,"15min":0.85799999999999998},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":11.374000000000001},"last":1.23},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3460000000000001,"15min":1.369},"min":{"1min":0.97899999999999998,"5min":0.94999999999999996,"15min":0.875},"max":{"1min":1.855,"5min":1.9810000000000001,"15min":2.3319999999999999},"last":1.173}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":27070464,"data_stored":27061270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22872064,"data_stored":22866980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12591104,"data_stored":12583584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":37752832,"data_stored":37749318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12599296,"data_stored":12584256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":25784320,"data_stored":25774643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":13201408,"data_stored":13191763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-21T22:52:42.470 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-21T22:52:42.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.715+0000 7f060fc00640 1 Processor -- start 2024-01-21T22:52:42.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.715+0000 7f060fc00640 1 -- start start 2024-01-21T22:52:42.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.717+0000 7f060fc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f060815a250 0x7f060815a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:42.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.717+0000 7f060fc00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06080c61c0 con 0x7f0608153cc0 2024-01-21T22:52:42.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.717+0000 7f060fc00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0608159dc0 con 0x7f060806a6f0 2024-01-21T22:52:42.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.717+0000 7f060fc00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0608159b20 con 0x7f060815a250 2024-01-21T22:52:42.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.718+0000 7f05ffc00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f060815a250 0x7f060815a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:42.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.718+0000 7f060dc00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f060806a6f0 0x7f06081644a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:60466/0 (socket says 172.21.15.138:60466) 2024-01-21T22:52:42.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.718+0000 7f060dc00640 1 -- 172.21.15.138:0/1121477694 learned_addr learned my addr 172.21.15.138:0/1121477694 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:52:42.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.722+0000 7f05ffc00640 1 -- 172.21.15.138:0/1121477694 >> v1:172.21.15.152:6789/0 conn(0x7f060806a6f0 legacy=0x7f06081644a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:42.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.722+0000 7f05ffc00640 1 -- 172.21.15.138:0/1121477694 >> v1:172.21.15.138:6789/0 conn(0x7f0608153cc0 legacy=0x7f06080c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:42.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.722+0000 7f05ffc00640 1 -- 172.21.15.138:0/1121477694 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0608071270 con 0x7f060815a250 2024-01-21T22:52:42.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.726+0000 7f05ffc00640 1 --2- 172.21.15.138:0/1121477694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f060815a250 0x7f060815a680 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f05f80097a0 tx=0x7f05f802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bbfed7cbc040870 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:42.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.730+0000 7f0607e00640 1 -- 172.21.15.138:0/1121477694 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f8031830 con 0x7f060815a250 2024-01-21T22:52:42.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.730+0000 7f0607e00640 1 -- 172.21.15.138:0/1121477694 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f8031990 con 0x7f060815a250 2024-01-21T22:52:42.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.730+0000 7f0607e00640 1 -- 172.21.15.138:0/1121477694 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f8031d20 con 0x7f060815a250 2024-01-21T22:52:42.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.731+0000 7f060fc00640 1 -- 172.21.15.138:0/1121477694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f060815a250 msgr2=0x7f060815a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:42.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.731+0000 7f060fc00640 1 --2- 172.21.15.138:0/1121477694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f060815a250 0x7f060815a680 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f05f80097a0 tx=0x7f05f802b8d0 comp rx=0 tx=0).stop 2024-01-21T22:52:42.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.731+0000 7f060fc00640 1 -- 172.21.15.138:0/1121477694 shutdown_connections 2024-01-21T22:52:42.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.731+0000 7f060fc00640 1 --2- 172.21.15.138:0/1121477694 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f060815a250 0x7f060815a680 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:42.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.731+0000 7f060fc00640 1 -- 172.21.15.138:0/1121477694 >> 172.21.15.138:0/1121477694 conn(0x7f06080cb0b0 msgr2=0x7f06080cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:42.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.731+0000 7f060fc00640 1 -- 172.21.15.138:0/1121477694 shutdown_connections 2024-01-21T22:52:42.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.731+0000 7f060fc00640 1 -- 172.21.15.138:0/1121477694 wait complete. 2024-01-21T22:52:42.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.733+0000 7f060fc00640 1 Processor -- start 2024-01-21T22:52:42.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.733+0000 7f060fc00640 1 -- start start 2024-01-21T22:52:42.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.734+0000 7f060fc00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f060806a6f0 0x7f06080c4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:42.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.734+0000 7f060fc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0608153cc0 0x7f060815cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:42.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.734+0000 7f060fc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f060815a250 0x7f06080c2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:42.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.734+0000 7f060fc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060815b9d0 con 0x7f060815a250 2024-01-21T22:52:42.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.734+0000 7f060fc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060815cce0 con 0x7f0608153cc0 2024-01-21T22:52:42.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.734+0000 7f060fc00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060815aca0 con 0x7f060806a6f0 2024-01-21T22:52:42.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.735+0000 7f05ffc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0608153cc0 0x7f060815cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:42.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.735+0000 7f060d200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f060806a6f0 0x7f06080c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:42.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.735+0000 7f05ffc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0608153cc0 0x7f060815cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:48198/0 (socket says 172.21.15.138:48198) 2024-01-21T22:52:42.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.735+0000 7f05ffc00640 1 -- 172.21.15.138:0/1406916703 learned_addr learned my addr 172.21.15.138:0/1406916703 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:42.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.739+0000 7f060d200640 1 -- 172.21.15.138:0/1406916703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0608153cc0 msgr2=0x7f060815cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:42.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.739+0000 7f060d200640 1 --2- 172.21.15.138:0/1406916703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0608153cc0 0x7f060815cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:42.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.739+0000 7f060d200640 1 -- 172.21.15.138:0/1406916703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f060815a250 msgr2=0x7f06080c2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:42.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.739+0000 7f060d200640 1 --2- 172.21.15.138:0/1406916703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f060815a250 0x7f06080c2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:42.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.739+0000 7f060d200640 1 -- 172.21.15.138:0/1406916703 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f8009400 con 0x7f060806a6f0 2024-01-21T22:52:42.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.741+0000 7f060d200640 1 --2- 172.21.15.138:0/1406916703 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f060806a6f0 0x7f06080c4860 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f05f000c440 tx=0x7f05f000c840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:42.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.743+0000 7f0605600640 1 -- 172.21.15.138:0/1406916703 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f00035f0 con 0x7f060806a6f0 2024-01-21T22:52:42.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.744+0000 7f060fc00640 1 -- 172.21.15.138:0/1406916703 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060815b540 con 0x7f060806a6f0 2024-01-21T22:52:42.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.744+0000 7f060fc00640 1 -- 172.21.15.138:0/1406916703 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060815db50 con 0x7f060806a6f0 2024-01-21T22:52:42.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.745+0000 7f060fc00640 1 -- 172.21.15.138:0/1406916703 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f060804eaa0 con 0x7f060806a6f0 2024-01-21T22:52:42.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.746+0000 7f0605600640 1 -- 172.21.15.138:0/1406916703 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f0003750 con 0x7f060806a6f0 2024-01-21T22:52:42.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.746+0000 7f0605600640 1 -- 172.21.15.138:0/1406916703 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f0003ae0 con 0x7f060806a6f0 2024-01-21T22:52:42.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.766+0000 7f0605600640 1 -- 172.21.15.138:0/1406916703 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f05f0003c40 con 0x7f060806a6f0 2024-01-21T22:52:42.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.767+0000 7f0605600640 1 --2- 172.21.15.138:0/1406916703 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f05e006b360 0x7f05e006d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:42.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.767+0000 7f05ffc00640 1 --2- 172.21.15.138:0/1406916703 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f05e006b360 0x7f05e006d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:42.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.768+0000 7f0605600640 1 -- 172.21.15.138:0/1406916703 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f05f00583c0 con 0x7f060806a6f0 2024-01-21T22:52:42.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:42.768+0000 7f05ffc00640 1 --2- 172.21.15.138:0/1406916703 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f05e006b360 0x7f05e006d820 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f05f80097a0 tx=0x7f05f803c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:43.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.007+0000 7f0605600640 1 -- 172.21.15.138:0/1406916703 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f05f0054700 con 0x7f060806a6f0 2024-01-21T22:52:43.144 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:52:43.152 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:52:43.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.199+0000 7f060fc00640 1 -- 172.21.15.138:0/1406916703 --> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0608071ad0 con 0x7f05e006b360 2024-01-21T22:52:43.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.207+0000 7f0605600640 1 -- 172.21.15.138:0/1406916703 <== mgr.4111 v2:172.21.15.152:6800/314625564 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289589 (secure 0 0 0) 0x7f05f8044400 con 0x7f05e006b360 2024-01-21T22:52:43.212 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:52:43.213 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-01-21T22:52:43.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.211+0000 7f060fc00640 1 -- 172.21.15.138:0/1406916703 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f05e006b360 msgr2=0x7f05e006d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:43.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.211+0000 7f060fc00640 1 --2- 172.21.15.138:0/1406916703 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f05e006b360 0x7f05e006d820 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f05f80097a0 tx=0x7f05f803c040 comp rx=0 tx=0).stop 2024-01-21T22:52:43.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.211+0000 7f060fc00640 1 -- 172.21.15.138:0/1406916703 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f060806a6f0 msgr2=0x7f06080c4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:43.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.211+0000 7f060fc00640 1 --2- 172.21.15.138:0/1406916703 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f060806a6f0 0x7f06080c4860 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f05f000c440 tx=0x7f05f000c840 comp rx=0 tx=0).stop 2024-01-21T22:52:43.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.211+0000 7f060fc00640 1 -- 172.21.15.138:0/1406916703 shutdown_connections 2024-01-21T22:52:43.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.211+0000 7f060fc00640 1 --2- 172.21.15.138:0/1406916703 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f05e006b360 0x7f05e006d820 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:43.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.211+0000 7f060fc00640 1 --2- 172.21.15.138:0/1406916703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f060815a250 0x7f06080c2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:43.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.211+0000 7f060fc00640 1 --2- 172.21.15.138:0/1406916703 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f0608153cc0 0x7f060815cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:43.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.211+0000 7f060fc00640 1 --2- 172.21.15.138:0/1406916703 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f060806a6f0 0x7f06080c4860 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:43.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.211+0000 7f060fc00640 1 -- 172.21.15.138:0/1406916703 >> 172.21.15.138:0/1406916703 conn(0x7f06080cb0b0 msgr2=0x7f0608150c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:43.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.211+0000 7f060fc00640 1 -- 172.21.15.138:0/1406916703 shutdown_connections 2024-01-21T22:52:43.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.211+0000 7f060fc00640 1 -- 172.21.15.138:0/1406916703 wait complete. 2024-01-21T22:52:43.231 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":858,"stamp":"2024-01-21T22:52:41.537774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":82787731,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":1693,"num_read_kb":212795,"num_write":26807,"num_write_kb":599374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27620,"ondisk_log_size":27620,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":606176,"kb_used_data":163808,"kb_used_omap":12,"kb_used_meta":442291,"kb_avail":754368544,"statfs":{"total":773094113280,"available":772473389056,"internally_reserved":0,"allocated":167739392,"data_stored":166350990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":452906545},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":272532,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":10,"num_write":87,"num_write_kb":322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002650"},"pg_stats":[{"pgid":"3.3f","version":"25'157","reported_seq":192,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.276848+0000","last_change":"2024-01-21T22:25:04.888366+0000","last_active":"2024-01-21T22:52:19.276848+0000","last_peered":"2024-01-21T22:52:19.276848+0000","last_clean":"2024-01-21T22:52:19.276848+0000","last_became_active":"2024-01-21T22:25:04.888060+0000","last_became_peered":"2024-01-21T22:25:04.888060+0000","last_unstale":"2024-01-21T22:52:19.276848+0000","last_undegraded":"2024-01-21T22:52:19.276848+0000","last_fullsized":"2024-01-21T22:52:19.276848+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:32:13.512213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":115,"num_write_kb":6171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"25'166","reported_seq":206,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.991866+0000","last_change":"2024-01-21T22:25:04.843704+0000","last_active":"2024-01-21T22:52:19.991866+0000","last_peered":"2024-01-21T22:52:19.991866+0000","last_clean":"2024-01-21T22:52:19.991866+0000","last_became_active":"2024-01-21T22:25:04.843471+0000","last_became_peered":"2024-01-21T22:25:04.843471+0000","last_unstale":"2024-01-21T22:52:19.991866+0000","last_undegraded":"2024-01-21T22:52:19.991866+0000","last_fullsized":"2024-01-21T22:52:19.991866+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:34:24.822768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":129,"num_write_kb":4796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"25'195","reported_seq":234,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.221491+0000","last_change":"2024-01-21T22:25:04.858021+0000","last_active":"2024-01-21T22:52:20.221491+0000","last_peered":"2024-01-21T22:52:20.221491+0000","last_clean":"2024-01-21T22:52:20.221491+0000","last_became_active":"2024-01-21T22:25:04.857685+0000","last_became_peered":"2024-01-21T22:25:04.857685+0000","last_unstale":"2024-01-21T22:52:20.221491+0000","last_undegraded":"2024-01-21T22:52:20.221491+0000","last_fullsized":"2024-01-21T22:52:20.221491+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:29:02.950903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":157,"num_write_kb":6223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"25'172","reported_seq":207,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.280308+0000","last_change":"2024-01-21T22:25:04.858941+0000","last_active":"2024-01-21T22:52:19.280308+0000","last_peered":"2024-01-21T22:52:19.280308+0000","last_clean":"2024-01-21T22:52:19.280308+0000","last_became_active":"2024-01-21T22:25:04.858278+0000","last_became_peered":"2024-01-21T22:25:04.858278+0000","last_unstale":"2024-01-21T22:52:19.280308+0000","last_undegraded":"2024-01-21T22:52:19.280308+0000","last_fullsized":"2024-01-21T22:52:19.280308+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T10:03:50.093635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":135,"num_write_kb":19244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"25'177","reported_seq":214,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.448004+0000","last_change":"2024-01-21T22:25:04.881014+0000","last_active":"2024-01-21T22:52:23.448004+0000","last_peered":"2024-01-21T22:52:23.448004+0000","last_clean":"2024-01-21T22:52:23.448004+0000","last_became_active":"2024-01-21T22:25:04.878457+0000","last_became_peered":"2024-01-21T22:25:04.878457+0000","last_unstale":"2024-01-21T22:52:23.448004+0000","last_undegraded":"2024-01-21T22:52:23.448004+0000","last_fullsized":"2024-01-21T22:52:23.448004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:34:24.822768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":119,"num_write_kb":7041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"25'193","reported_seq":221,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.279603+0000","last_change":"2024-01-21T22:25:04.859776+0000","last_active":"2024-01-21T22:52:19.279603+0000","last_peered":"2024-01-21T22:52:19.279603+0000","last_clean":"2024-01-21T22:52:19.279603+0000","last_became_active":"2024-01-21T22:25:04.858731+0000","last_became_peered":"2024-01-21T22:25:04.858731+0000","last_unstale":"2024-01-21T22:52:19.279603+0000","last_undegraded":"2024-01-21T22:52:19.279603+0000","last_fullsized":"2024-01-21T22:52:19.279603+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:01:30.091588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":148,"num_write_kb":2956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"25'207","reported_seq":251,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.135091+0000","last_change":"2024-01-21T22:25:04.881392+0000","last_active":"2024-01-21T22:52:20.135091+0000","last_peered":"2024-01-21T22:52:20.135091+0000","last_clean":"2024-01-21T22:52:20.135091+0000","last_became_active":"2024-01-21T22:25:04.881090+0000","last_became_peered":"2024-01-21T22:25:04.881090+0000","last_unstale":"2024-01-21T22:52:20.135091+0000","last_undegraded":"2024-01-21T22:52:20.135091+0000","last_fullsized":"2024-01-21T22:52:20.135091+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:25:50.380424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":4572,"num_write":149,"num_write_kb":5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"25'165","reported_seq":204,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.138549+0000","last_change":"2024-01-21T22:25:04.879186+0000","last_active":"2024-01-21T22:52:20.138549+0000","last_peered":"2024-01-21T22:52:20.138549+0000","last_clean":"2024-01-21T22:52:20.138549+0000","last_became_active":"2024-01-21T22:25:04.878824+0000","last_became_peered":"2024-01-21T22:25:04.878824+0000","last_unstale":"2024-01-21T22:52:20.138549+0000","last_undegraded":"2024-01-21T22:52:20.138549+0000","last_fullsized":"2024-01-21T22:52:20.138549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:31:25.985811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":137,"num_write_kb":11909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"25'151","reported_seq":184,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.378252+0000","last_change":"2024-01-21T22:25:04.886391+0000","last_active":"2024-01-21T22:52:23.378252+0000","last_peered":"2024-01-21T22:52:23.378252+0000","last_clean":"2024-01-21T22:52:23.378252+0000","last_became_active":"2024-01-21T22:25:04.884979+0000","last_became_peered":"2024-01-21T22:25:04.884979+0000","last_unstale":"2024-01-21T22:52:23.378252+0000","last_undegraded":"2024-01-21T22:52:23.378252+0000","last_fullsized":"2024-01-21T22:52:23.378252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:36:36.009632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":96,"num_write_kb":3924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"25'163","reported_seq":202,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.131949+0000","last_change":"2024-01-21T22:25:04.888126+0000","last_active":"2024-01-21T22:52:20.131949+0000","last_peered":"2024-01-21T22:52:20.131949+0000","last_clean":"2024-01-21T22:52:20.131949+0000","last_became_active":"2024-01-21T22:25:04.887631+0000","last_became_peered":"2024-01-21T22:25:04.887631+0000","last_unstale":"2024-01-21T22:52:20.131949+0000","last_undegraded":"2024-01-21T22:52:20.131949+0000","last_fullsized":"2024-01-21T22:52:20.131949+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:00:02.684745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":122,"num_write_kb":9871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"25'186","reported_seq":220,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.221571+0000","last_change":"2024-01-21T22:25:04.885677+0000","last_active":"2024-01-21T22:52:20.221571+0000","last_peered":"2024-01-21T22:52:20.221571+0000","last_clean":"2024-01-21T22:52:20.221571+0000","last_became_active":"2024-01-21T22:25:04.884697+0000","last_became_peered":"2024-01-21T22:25:04.884697+0000","last_unstale":"2024-01-21T22:52:20.221571+0000","last_undegraded":"2024-01-21T22:52:20.221571+0000","last_fullsized":"2024-01-21T22:52:20.221571+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:44:23.749631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":143,"num_write_kb":7172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"25'139","reported_seq":176,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:07.428734+0000","last_change":"2024-01-21T22:25:04.886799+0000","last_active":"2024-01-21T22:52:07.428734+0000","last_peered":"2024-01-21T22:52:07.428734+0000","last_clean":"2024-01-21T22:52:07.428734+0000","last_became_active":"2024-01-21T22:25:04.886387+0000","last_became_peered":"2024-01-21T22:25:04.886387+0000","last_unstale":"2024-01-21T22:52:07.428734+0000","last_undegraded":"2024-01-21T22:52:07.428734+0000","last_fullsized":"2024-01-21T22:52:07.428734+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:42:39.977125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":92,"num_write_kb":3768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"25'159","reported_seq":192,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.564136+0000","last_change":"2024-01-21T22:25:04.888103+0000","last_active":"2024-01-21T22:52:19.564136+0000","last_peered":"2024-01-21T22:52:19.564136+0000","last_clean":"2024-01-21T22:52:19.564136+0000","last_became_active":"2024-01-21T22:25:04.887908+0000","last_became_peered":"2024-01-21T22:25:04.887908+0000","last_unstale":"2024-01-21T22:52:19.564136+0000","last_undegraded":"2024-01-21T22:52:19.564136+0000","last_fullsized":"2024-01-21T22:52:19.564136+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:23:52.444277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":122,"num_write_kb":8957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"25'167","reported_seq":206,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.374792+0000","last_change":"2024-01-21T22:25:04.842750+0000","last_active":"2024-01-21T22:52:23.374792+0000","last_peered":"2024-01-21T22:52:23.374792+0000","last_clean":"2024-01-21T22:52:23.374792+0000","last_became_active":"2024-01-21T22:25:04.842512+0000","last_became_peered":"2024-01-21T22:25:04.842512+0000","last_unstale":"2024-01-21T22:52:23.374792+0000","last_undegraded":"2024-01-21T22:52:23.374792+0000","last_fullsized":"2024-01-21T22:52:23.374792+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:40:05.554704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":120,"num_write_kb":6061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"25'163","reported_seq":219,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.138574+0000","last_change":"2024-01-21T22:25:04.876592+0000","last_active":"2024-01-21T22:52:20.138574+0000","last_peered":"2024-01-21T22:52:20.138574+0000","last_clean":"2024-01-21T22:52:20.138574+0000","last_became_active":"2024-01-21T22:25:04.876305+0000","last_became_peered":"2024-01-21T22:25:04.876305+0000","last_unstale":"2024-01-21T22:52:20.138574+0000","last_undegraded":"2024-01-21T22:52:20.138574+0000","last_fullsized":"2024-01-21T22:52:20.138574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:32.177562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":7620,"num_write":119,"num_write_kb":10742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"25'200","reported_seq":246,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.273197+0000","last_change":"2024-01-21T22:25:04.878038+0000","last_active":"2024-01-21T22:52:23.273197+0000","last_peered":"2024-01-21T22:52:23.273197+0000","last_clean":"2024-01-21T22:52:23.273197+0000","last_became_active":"2024-01-21T22:25:04.877716+0000","last_became_peered":"2024-01-21T22:25:04.877716+0000","last_unstale":"2024-01-21T22:52:23.273197+0000","last_undegraded":"2024-01-21T22:52:23.273197+0000","last_fullsized":"2024-01-21T22:52:23.273197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:32:13.512213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":5080,"num_write":159,"num_write_kb":7672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"25'175","reported_seq":212,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.145332+0000","last_change":"2024-01-21T22:25:04.890537+0000","last_active":"2024-01-21T22:52:20.145332+0000","last_peered":"2024-01-21T22:52:20.145332+0000","last_clean":"2024-01-21T22:52:20.145332+0000","last_became_active":"2024-01-21T22:25:04.890298+0000","last_became_peered":"2024-01-21T22:25:04.890298+0000","last_unstale":"2024-01-21T22:52:20.145332+0000","last_undegraded":"2024-01-21T22:52:20.145332+0000","last_fullsized":"2024-01-21T22:52:20.145332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:23:57.435028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":136,"num_write_kb":7248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"25'149","reported_seq":184,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.940326+0000","last_change":"2024-01-21T22:25:04.712868+0000","last_active":"2024-01-21T22:52:19.940326+0000","last_peered":"2024-01-21T22:52:19.940326+0000","last_clean":"2024-01-21T22:52:19.940326+0000","last_became_active":"2024-01-21T22:25:04.712616+0000","last_became_peered":"2024-01-21T22:25:04.712616+0000","last_unstale":"2024-01-21T22:52:19.940326+0000","last_undegraded":"2024-01-21T22:52:19.940326+0000","last_fullsized":"2024-01-21T22:52:19.940326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:50:41.306582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":102,"num_write_kb":4431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"25'156","reported_seq":196,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.470695+0000","last_change":"2024-01-21T22:25:04.879328+0000","last_active":"2024-01-21T22:52:19.470695+0000","last_peered":"2024-01-21T22:52:19.470695+0000","last_clean":"2024-01-21T22:52:19.470695+0000","last_became_active":"2024-01-21T22:25:04.879011+0000","last_became_peered":"2024-01-21T22:25:04.879011+0000","last_unstale":"2024-01-21T22:52:19.470695+0000","last_undegraded":"2024-01-21T22:52:19.470695+0000","last_fullsized":"2024-01-21T22:52:19.470695+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:31:23.569672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":108,"num_write_kb":6073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"25'235","reported_seq":276,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.448463+0000","last_change":"2024-01-21T22:25:04.885620+0000","last_active":"2024-01-21T22:52:23.448463+0000","last_peered":"2024-01-21T22:52:23.448463+0000","last_clean":"2024-01-21T22:52:23.448463+0000","last_became_active":"2024-01-21T22:25:04.884364+0000","last_became_peered":"2024-01-21T22:25:04.884364+0000","last_unstale":"2024-01-21T22:52:23.448463+0000","last_undegraded":"2024-01-21T22:52:23.448463+0000","last_fullsized":"2024-01-21T22:52:23.448463+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:13:04.601911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3556,"num_write":191,"num_write_kb":16584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"25'189","reported_seq":228,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.934988+0000","last_change":"2024-01-21T22:25:04.868371+0000","last_active":"2024-01-21T22:52:19.934988+0000","last_peered":"2024-01-21T22:52:19.934988+0000","last_clean":"2024-01-21T22:52:19.934988+0000","last_became_active":"2024-01-21T22:25:04.868196+0000","last_became_peered":"2024-01-21T22:25:04.868196+0000","last_unstale":"2024-01-21T22:52:19.934988+0000","last_undegraded":"2024-01-21T22:52:19.934988+0000","last_fullsized":"2024-01-21T22:52:19.934988+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":189,"log_dups_size":0,"ondisk_log_size":189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:05:41.950232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":139,"num_write_kb":3732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"25'193","reported_seq":250,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.563845+0000","last_change":"2024-01-21T22:25:04.879144+0000","last_active":"2024-01-21T22:52:19.563845+0000","last_peered":"2024-01-21T22:52:19.563845+0000","last_clean":"2024-01-21T22:52:19.563845+0000","last_became_active":"2024-01-21T22:25:04.878868+0000","last_became_peered":"2024-01-21T22:25:04.878868+0000","last_unstale":"2024-01-21T22:52:19.563845+0000","last_undegraded":"2024-01-21T22:52:19.563845+0000","last_fullsized":"2024-01-21T22:52:19.563845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:55:28.069411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":7620,"num_write":153,"num_write_kb":6732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"25'165","reported_seq":201,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.934401+0000","last_change":"2024-01-21T22:25:04.715246+0000","last_active":"2024-01-21T22:52:19.934401+0000","last_peered":"2024-01-21T22:52:19.934401+0000","last_clean":"2024-01-21T22:52:19.934401+0000","last_became_active":"2024-01-21T22:25:04.714969+0000","last_became_peered":"2024-01-21T22:25:04.714969+0000","last_unstale":"2024-01-21T22:52:19.934401+0000","last_undegraded":"2024-01-21T22:52:19.934401+0000","last_fullsized":"2024-01-21T22:52:19.934401+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:56:24.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":106,"num_write_kb":4123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"25'176","reported_seq":224,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.406727+0000","last_change":"2024-01-21T22:25:04.877902+0000","last_active":"2024-01-21T22:52:19.406727+0000","last_peered":"2024-01-21T22:52:19.406727+0000","last_clean":"2024-01-21T22:52:19.406727+0000","last_became_active":"2024-01-21T22:25:04.876963+0000","last_became_peered":"2024-01-21T22:25:04.876963+0000","last_unstale":"2024-01-21T22:52:19.406727+0000","last_undegraded":"2024-01-21T22:52:19.406727+0000","last_fullsized":"2024-01-21T22:52:19.406727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:30:48.864454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":5334,"num_write":137,"num_write_kb":8161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"25'182","reported_seq":232,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.412606+0000","last_change":"2024-01-21T22:25:04.716735+0000","last_active":"2024-01-21T22:52:23.412606+0000","last_peered":"2024-01-21T22:52:23.412606+0000","last_clean":"2024-01-21T22:52:23.412606+0000","last_became_active":"2024-01-21T22:25:04.716570+0000","last_became_peered":"2024-01-21T22:25:04.716570+0000","last_unstale":"2024-01-21T22:52:23.412606+0000","last_undegraded":"2024-01-21T22:52:23.412606+0000","last_fullsized":"2024-01-21T22:52:23.412606+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:44:55.158990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":5842,"num_write":145,"num_write_kb":5757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"25'161","reported_seq":202,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.395083+0000","last_change":"2024-01-21T22:25:04.886230+0000","last_active":"2024-01-21T22:52:23.395083+0000","last_peered":"2024-01-21T22:52:23.395083+0000","last_clean":"2024-01-21T22:52:23.395083+0000","last_became_active":"2024-01-21T22:25:04.885523+0000","last_became_peered":"2024-01-21T22:25:04.885523+0000","last_unstale":"2024-01-21T22:52:23.395083+0000","last_undegraded":"2024-01-21T22:52:23.395083+0000","last_fullsized":"2024-01-21T22:52:23.395083+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:06:02.934920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3556,"num_write":106,"num_write_kb":5820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"25'207","reported_seq":243,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.561247+0000","last_change":"2024-01-21T22:25:04.842692+0000","last_active":"2024-01-21T22:52:19.561247+0000","last_peered":"2024-01-21T22:52:19.561247+0000","last_clean":"2024-01-21T22:52:19.561247+0000","last_became_active":"2024-01-21T22:25:04.842429+0000","last_became_peered":"2024-01-21T22:25:04.842429+0000","last_unstale":"2024-01-21T22:52:19.561247+0000","last_undegraded":"2024-01-21T22:52:19.561247+0000","last_fullsized":"2024-01-21T22:52:19.561247+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:52:24.233709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":167,"num_write_kb":10319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"25'179","reported_seq":218,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.378526+0000","last_change":"2024-01-21T22:25:04.888296+0000","last_active":"2024-01-21T22:52:23.378526+0000","last_peered":"2024-01-21T22:52:23.378526+0000","last_clean":"2024-01-21T22:52:23.378526+0000","last_became_active":"2024-01-21T22:25:04.888025+0000","last_became_peered":"2024-01-21T22:25:04.888025+0000","last_unstale":"2024-01-21T22:52:23.378526+0000","last_undegraded":"2024-01-21T22:52:23.378526+0000","last_fullsized":"2024-01-21T22:52:23.378526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":179,"log_dups_size":0,"ondisk_log_size":179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:57:44.129490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":136,"num_write_kb":6753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"25'192","reported_seq":232,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.222625+0000","last_change":"2024-01-21T22:25:04.711908+0000","last_active":"2024-01-21T22:52:20.222625+0000","last_peered":"2024-01-21T22:52:20.222625+0000","last_clean":"2024-01-21T22:52:20.222625+0000","last_became_active":"2024-01-21T22:25:04.711654+0000","last_became_peered":"2024-01-21T22:25:04.711654+0000","last_unstale":"2024-01-21T22:52:20.222625+0000","last_undegraded":"2024-01-21T22:52:20.222625+0000","last_fullsized":"2024-01-21T22:52:20.222625+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:20:41.612265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":148,"num_write_kb":7328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"25'326","reported_seq":367,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.144991+0000","last_change":"2024-01-21T22:25:04.876757+0000","last_active":"2024-01-21T22:52:20.144991+0000","last_peered":"2024-01-21T22:52:20.144991+0000","last_clean":"2024-01-21T22:52:20.144991+0000","last_became_active":"2024-01-21T22:25:04.876012+0000","last_became_peered":"2024-01-21T22:25:04.876012+0000","last_unstale":"2024-01-21T22:52:20.144991+0000","last_undegraded":"2024-01-21T22:52:20.144991+0000","last_fullsized":"2024-01-21T22:52:20.144991+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:00:56.239773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":275,"num_write_kb":7140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"25'195","reported_seq":231,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.940456+0000","last_change":"2024-01-21T22:25:04.715277+0000","last_active":"2024-01-21T22:52:19.940456+0000","last_peered":"2024-01-21T22:52:19.940456+0000","last_clean":"2024-01-21T22:52:19.940456+0000","last_became_active":"2024-01-21T22:25:04.715034+0000","last_became_peered":"2024-01-21T22:25:04.715034+0000","last_unstale":"2024-01-21T22:52:19.940456+0000","last_undegraded":"2024-01-21T22:52:19.940456+0000","last_fullsized":"2024-01-21T22:52:19.940456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:55:28.069411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":140,"num_write_kb":6049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"25'201","reported_seq":233,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.010555+0000","last_change":"2024-01-21T22:25:04.844472+0000","last_active":"2024-01-21T22:52:20.010555+0000","last_peered":"2024-01-21T22:52:20.010555+0000","last_clean":"2024-01-21T22:52:20.010555+0000","last_became_active":"2024-01-21T22:25:04.843827+0000","last_became_peered":"2024-01-21T22:25:04.843827+0000","last_unstale":"2024-01-21T22:52:20.010555+0000","last_undegraded":"2024-01-21T22:52:20.010555+0000","last_fullsized":"2024-01-21T22:52:20.010555+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:58:10.552987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1016,"num_write":149,"num_write_kb":12027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"25'151","reported_seq":188,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.381267+0000","last_change":"2024-01-21T22:25:04.713764+0000","last_active":"2024-01-21T22:52:23.381267+0000","last_peered":"2024-01-21T22:52:23.381267+0000","last_clean":"2024-01-21T22:52:23.381267+0000","last_became_active":"2024-01-21T22:25:04.712814+0000","last_became_peered":"2024-01-21T22:25:04.712814+0000","last_unstale":"2024-01-21T22:52:23.381267+0000","last_undegraded":"2024-01-21T22:52:23.381267+0000","last_fullsized":"2024-01-21T22:52:23.381267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":151,"log_dups_size":0,"ondisk_log_size":151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:16:41.440927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":111,"num_write_kb":4009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"25'751","reported_seq":778,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.145057+0000","last_change":"2024-01-21T22:25:03.703079+0000","last_active":"2024-01-21T22:52:33.145057+0000","last_peered":"2024-01-21T22:52:33.145057+0000","last_clean":"2024-01-21T22:52:33.145057+0000","last_became_active":"2024-01-21T22:25:03.702776+0000","last_became_peered":"2024-01-21T22:25:03.702776+0000","last_unstale":"2024-01-21T22:52:33.145057+0000","last_undegraded":"2024-01-21T22:52:33.145057+0000","last_fullsized":"2024-01-21T22:52:33.145057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":751,"log_dups_size":0,"ondisk_log_size":751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:34:13.302149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":748,"num_write_kb":4569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"25'190","reported_seq":227,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.558673+0000","last_change":"2024-01-21T22:25:04.877158+0000","last_active":"2024-01-21T22:52:19.558673+0000","last_peered":"2024-01-21T22:52:19.558673+0000","last_clean":"2024-01-21T22:52:19.558673+0000","last_became_active":"2024-01-21T22:25:04.876920+0000","last_became_peered":"2024-01-21T22:25:04.876920+0000","last_unstale":"2024-01-21T22:52:19.558673+0000","last_undegraded":"2024-01-21T22:52:19.558673+0000","last_fullsized":"2024-01-21T22:52:19.558673+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:46:55.451173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":152,"num_write_kb":6158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"25'776","reported_seq":803,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.474218+0000","last_change":"2024-01-21T22:25:03.720178+0000","last_active":"2024-01-21T22:52:33.474218+0000","last_peered":"2024-01-21T22:52:33.474218+0000","last_clean":"2024-01-21T22:52:33.474218+0000","last_became_active":"2024-01-21T22:25:03.719909+0000","last_became_peered":"2024-01-21T22:25:03.719909+0000","last_unstale":"2024-01-21T22:52:33.474218+0000","last_undegraded":"2024-01-21T22:52:33.474218+0000","last_fullsized":"2024-01-21T22:52:33.474218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:44:53.910872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":769,"num_write_kb":4580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"25'3","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.462701+0000","last_change":"2024-01-21T22:25:03.723005+0000","last_active":"2024-01-21T22:52:28.462701+0000","last_peered":"2024-01-21T22:52:28.462701+0000","last_clean":"2024-01-21T22:52:28.462701+0000","last_became_active":"2024-01-21T22:25:03.722758+0000","last_became_peered":"2024-01-21T22:25:03.722758+0000","last_unstale":"2024-01-21T22:52:28.462701+0000","last_undegraded":"2024-01-21T22:52:28.462701+0000","last_fullsized":"2024-01-21T22:52:28.462701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:16:40.192808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"25'216","reported_seq":255,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.972006+0000","last_change":"2024-01-21T22:25:04.844030+0000","last_active":"2024-01-21T22:52:19.972006+0000","last_peered":"2024-01-21T22:52:19.972006+0000","last_clean":"2024-01-21T22:52:19.972006+0000","last_became_active":"2024-01-21T22:25:04.843714+0000","last_became_peered":"2024-01-21T22:25:04.843714+0000","last_unstale":"2024-01-21T22:52:19.972006+0000","last_undegraded":"2024-01-21T22:52:19.972006+0000","last_fullsized":"2024-01-21T22:52:19.972006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:25:50.380424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":174,"num_write_kb":5546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"25'1195","reported_seq":1222,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.505239+0000","last_change":"2024-01-21T22:25:03.702767+0000","last_active":"2024-01-21T22:52:23.505239+0000","last_peered":"2024-01-21T22:52:23.505239+0000","last_clean":"2024-01-21T22:52:23.505239+0000","last_became_active":"2024-01-21T22:25:03.702140+0000","last_became_peered":"2024-01-21T22:25:03.702140+0000","last_unstale":"2024-01-21T22:52:23.505239+0000","last_undegraded":"2024-01-21T22:52:23.505239+0000","last_fullsized":"2024-01-21T22:52:23.505239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:46:54.203055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1190,"num_write_kb":8787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"25'188","reported_seq":226,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.121546+0000","last_change":"2024-01-21T22:25:04.867173+0000","last_active":"2024-01-21T22:52:20.121546+0000","last_peered":"2024-01-21T22:52:20.121546+0000","last_clean":"2024-01-21T22:52:20.121546+0000","last_became_active":"2024-01-21T22:25:04.866945+0000","last_became_peered":"2024-01-21T22:25:04.866945+0000","last_unstale":"2024-01-21T22:52:20.121546+0000","last_undegraded":"2024-01-21T22:52:20.121546+0000","last_fullsized":"2024-01-21T22:52:20.121546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:15:41.725132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":136,"num_write_kb":8996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.462303+0000","last_change":"2024-01-21T22:25:03.715128+0000","last_active":"2024-01-21T22:52:23.462303+0000","last_peered":"2024-01-21T22:52:23.462303+0000","last_clean":"2024-01-21T22:52:23.462303+0000","last_became_active":"2024-01-21T22:25:03.714039+0000","last_became_peered":"2024-01-21T22:25:03.714039+0000","last_unstale":"2024-01-21T22:52:23.462303+0000","last_undegraded":"2024-01-21T22:52:23.462303+0000","last_fullsized":"2024-01-21T22:52:23.462303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:09:09.988399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"25'200","reported_seq":238,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.220935+0000","last_change":"2024-01-21T22:25:04.886534+0000","last_active":"2024-01-21T22:52:20.220935+0000","last_peered":"2024-01-21T22:52:20.220935+0000","last_clean":"2024-01-21T22:52:20.220935+0000","last_became_active":"2024-01-21T22:25:04.885963+0000","last_became_peered":"2024-01-21T22:25:04.885963+0000","last_unstale":"2024-01-21T22:52:20.220935+0000","last_undegraded":"2024-01-21T22:52:20.220935+0000","last_fullsized":"2024-01-21T22:52:20.220935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:44:31.867220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":149,"num_write_kb":6741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"25'4","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.470132+0000","last_change":"2024-01-21T22:25:03.718237+0000","last_active":"2024-01-21T22:52:28.470132+0000","last_peered":"2024-01-21T22:52:28.470132+0000","last_clean":"2024-01-21T22:52:28.470132+0000","last_became_active":"2024-01-21T22:25:03.717937+0000","last_became_peered":"2024-01-21T22:25:03.717937+0000","last_unstale":"2024-01-21T22:52:28.470132+0000","last_undegraded":"2024-01-21T22:52:28.470132+0000","last_fullsized":"2024-01-21T22:52:28.470132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:31:24.737692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"25'186","reported_seq":222,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.137695+0000","last_change":"2024-01-21T22:25:04.882743+0000","last_active":"2024-01-21T22:52:20.137695+0000","last_peered":"2024-01-21T22:52:20.137695+0000","last_clean":"2024-01-21T22:52:20.137695+0000","last_became_active":"2024-01-21T22:25:04.882478+0000","last_became_peered":"2024-01-21T22:25:04.882478+0000","last_unstale":"2024-01-21T22:52:20.137695+0000","last_undegraded":"2024-01-21T22:52:20.137695+0000","last_fullsized":"2024-01-21T22:52:20.137695+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:42:39.977125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2540,"num_write":148,"num_write_kb":5415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"25'467","reported_seq":494,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.492525+0000","last_change":"2024-01-21T22:25:03.703672+0000","last_active":"2024-01-21T22:52:33.492525+0000","last_peered":"2024-01-21T22:52:33.492525+0000","last_clean":"2024-01-21T22:52:33.492525+0000","last_became_active":"2024-01-21T22:25:03.703426+0000","last_became_peered":"2024-01-21T22:25:03.703426+0000","last_unstale":"2024-01-21T22:52:33.492525+0000","last_undegraded":"2024-01-21T22:52:33.492525+0000","last_fullsized":"2024-01-21T22:52:33.492525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:41:51.793206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":466,"num_write_kb":463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"25'204","reported_seq":252,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.603931+0000","last_change":"2024-01-21T22:25:04.887306+0000","last_active":"2024-01-21T22:52:19.603931+0000","last_peered":"2024-01-21T22:52:19.603931+0000","last_clean":"2024-01-21T22:52:19.603931+0000","last_became_active":"2024-01-21T22:25:04.886970+0000","last_became_peered":"2024-01-21T22:25:04.886970+0000","last_unstale":"2024-01-21T22:52:19.603931+0000","last_undegraded":"2024-01-21T22:52:19.603931+0000","last_fullsized":"2024-01-21T22:52:19.603931+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:59:45.239426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":5588,"num_write":173,"num_write_kb":7144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"25'878","reported_seq":905,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.282057+0000","last_change":"2024-01-21T22:25:03.714102+0000","last_active":"2024-01-21T22:52:19.282057+0000","last_peered":"2024-01-21T22:52:19.282057+0000","last_clean":"2024-01-21T22:52:19.282057+0000","last_became_active":"2024-01-21T22:25:03.713717+0000","last_became_peered":"2024-01-21T22:25:03.713717+0000","last_unstale":"2024-01-21T22:52:19.282057+0000","last_undegraded":"2024-01-21T22:52:19.282057+0000","last_fullsized":"2024-01-21T22:52:19.282057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:32:15.181238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":877,"num_write_kb":4705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"25'163","reported_seq":207,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.933099+0000","last_change":"2024-01-21T22:25:04.881307+0000","last_active":"2024-01-21T22:52:19.933099+0000","last_peered":"2024-01-21T22:52:19.933099+0000","last_clean":"2024-01-21T22:52:19.933099+0000","last_became_active":"2024-01-21T22:25:04.880472+0000","last_became_peered":"2024-01-21T22:25:04.880472+0000","last_unstale":"2024-01-21T22:52:19.933099+0000","last_undegraded":"2024-01-21T22:52:19.933099+0000","last_fullsized":"2024-01-21T22:52:19.933099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:29:02.950903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":4572,"num_write":122,"num_write_kb":6796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"25'1290","reported_seq":1317,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.495049+0000","last_change":"2024-01-21T22:25:03.721825+0000","last_active":"2024-01-21T22:52:33.495049+0000","last_peered":"2024-01-21T22:52:33.495049+0000","last_clean":"2024-01-21T22:52:33.495049+0000","last_became_active":"2024-01-21T22:25:03.721502+0000","last_became_peered":"2024-01-21T22:25:03.721502+0000","last_unstale":"2024-01-21T22:52:33.495049+0000","last_undegraded":"2024-01-21T22:52:33.495049+0000","last_fullsized":"2024-01-21T22:52:33.495049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1290,"log_dups_size":0,"ondisk_log_size":1290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:09:09.988399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1286,"num_write_kb":8849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"25'155","reported_seq":186,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.935419+0000","last_change":"2024-01-21T22:25:04.877455+0000","last_active":"2024-01-21T22:52:19.935419+0000","last_peered":"2024-01-21T22:52:19.935419+0000","last_clean":"2024-01-21T22:52:19.935419+0000","last_became_active":"2024-01-21T22:25:04.877257+0000","last_became_peered":"2024-01-21T22:25:04.877257+0000","last_unstale":"2024-01-21T22:52:19.935419+0000","last_undegraded":"2024-01-21T22:52:19.935419+0000","last_fullsized":"2024-01-21T22:52:19.935419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:34:14.550267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1270,"num_write":117,"num_write_kb":4465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"25'4","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.497453+0000","last_change":"2024-01-21T22:25:03.706077+0000","last_active":"2024-01-21T22:52:28.497453+0000","last_peered":"2024-01-21T22:52:28.497453+0000","last_clean":"2024-01-21T22:52:28.497453+0000","last_became_active":"2024-01-21T22:25:03.702798+0000","last_became_peered":"2024-01-21T22:25:03.702798+0000","last_unstale":"2024-01-21T22:52:28.497453+0000","last_undegraded":"2024-01-21T22:52:28.497453+0000","last_fullsized":"2024-01-21T22:52:28.497453+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:20:40.364147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"25'216","reported_seq":251,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.108670+0000","last_change":"2024-01-21T22:25:04.890636+0000","last_active":"2024-01-21T22:52:19.108670+0000","last_peered":"2024-01-21T22:52:19.108670+0000","last_clean":"2024-01-21T22:52:19.108670+0000","last_became_active":"2024-01-21T22:25:04.890271+0000","last_became_peered":"2024-01-21T22:25:04.890271+0000","last_unstale":"2024-01-21T22:52:19.108670+0000","last_undegraded":"2024-01-21T22:52:19.108670+0000","last_fullsized":"2024-01-21T22:52:19.108670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:15:46.754395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":174,"num_write_kb":31551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"25'4","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.470849+0000","last_change":"2024-01-21T22:25:03.717202+0000","last_active":"2024-01-21T22:52:28.470849+0000","last_peered":"2024-01-21T22:52:28.470849+0000","last_clean":"2024-01-21T22:52:28.470849+0000","last_became_active":"2024-01-21T22:25:03.716846+0000","last_became_peered":"2024-01-21T22:25:03.716846+0000","last_unstale":"2024-01-21T22:52:28.470849+0000","last_undegraded":"2024-01-21T22:52:28.470849+0000","last_fullsized":"2024-01-21T22:52:28.470849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:10.904923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"25'163","reported_seq":196,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:07.183268+0000","last_change":"2024-01-21T22:25:04.715053+0000","last_active":"2024-01-21T22:52:07.183268+0000","last_peered":"2024-01-21T22:52:07.183268+0000","last_clean":"2024-01-21T22:52:07.183268+0000","last_became_active":"2024-01-21T22:25:04.713566+0000","last_became_peered":"2024-01-21T22:25:04.713566+0000","last_unstale":"2024-01-21T22:52:07.183268+0000","last_undegraded":"2024-01-21T22:52:07.183268+0000","last_fullsized":"2024-01-21T22:52:07.183268+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:50:41.306582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":119,"num_write_kb":3558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"25'9","reported_seq":36,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:50:43.264719+0000","last_change":"2024-01-21T22:25:03.703017+0000","last_active":"2024-01-21T22:50:43.264719+0000","last_peered":"2024-01-21T22:50:43.264719+0000","last_clean":"2024-01-21T22:50:43.264719+0000","last_became_active":"2024-01-21T22:25:03.702742+0000","last_became_peered":"2024-01-21T22:25:03.702742+0000","last_unstale":"2024-01-21T22:50:43.264719+0000","last_undegraded":"2024-01-21T22:50:43.264719+0000","last_fullsized":"2024-01-21T22:50:43.264719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:42:03.386594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"25'190","reported_seq":223,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.394334+0000","last_change":"2024-01-21T22:25:04.876507+0000","last_active":"2024-01-21T22:52:23.394334+0000","last_peered":"2024-01-21T22:52:23.394334+0000","last_clean":"2024-01-21T22:52:23.394334+0000","last_became_active":"2024-01-21T22:25:04.876169+0000","last_became_peered":"2024-01-21T22:25:04.876169+0000","last_unstale":"2024-01-21T22:52:23.394334+0000","last_undegraded":"2024-01-21T22:52:23.394334+0000","last_fullsized":"2024-01-21T22:52:23.394334+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:12:11.958115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":142,"num_write_kb":8431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"25'7","reported_seq":35,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.467670+0000","last_change":"2024-01-21T22:25:03.720257+0000","last_active":"2024-01-21T22:52:28.467670+0000","last_peered":"2024-01-21T22:52:28.467670+0000","last_clean":"2024-01-21T22:52:28.467670+0000","last_became_active":"2024-01-21T22:25:03.720009+0000","last_became_peered":"2024-01-21T22:25:03.720009+0000","last_unstale":"2024-01-21T22:52:28.467670+0000","last_undegraded":"2024-01-21T22:52:28.467670+0000","last_fullsized":"2024-01-21T22:52:28.467670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:37:56.569677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"25'148","reported_seq":196,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.221225+0000","last_change":"2024-01-21T22:25:04.883112+0000","last_active":"2024-01-21T22:52:20.221225+0000","last_peered":"2024-01-21T22:52:20.221225+0000","last_clean":"2024-01-21T22:52:20.221225+0000","last_became_active":"2024-01-21T22:25:04.882767+0000","last_became_peered":"2024-01-21T22:25:04.882767+0000","last_unstale":"2024-01-21T22:52:20.221225+0000","last_undegraded":"2024-01-21T22:52:20.221225+0000","last_fullsized":"2024-01-21T22:52:20.221225+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:18:03.439738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":5334,"num_write":115,"num_write_kb":10137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"25'300","reported_seq":328,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.539211+0000","last_change":"2024-01-21T22:25:03.703516+0000","last_active":"2024-01-21T22:52:33.539211+0000","last_peered":"2024-01-21T22:52:33.539211+0000","last_clean":"2024-01-21T22:52:33.539211+0000","last_became_active":"2024-01-21T22:25:03.703256+0000","last_became_peered":"2024-01-21T22:25:03.703256+0000","last_unstale":"2024-01-21T22:52:33.539211+0000","last_undegraded":"2024-01-21T22:52:33.539211+0000","last_fullsized":"2024-01-21T22:52:33.539211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:57:42.881371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":296,"num_write_kb":4241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"25'157","reported_seq":190,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.449441+0000","last_change":"2024-01-21T22:25:38.276745+0000","last_active":"2024-01-21T22:52:23.449441+0000","last_peered":"2024-01-21T22:52:23.449441+0000","last_clean":"2024-01-21T22:52:23.449441+0000","last_became_active":"2024-01-21T22:25:38.276512+0000","last_became_peered":"2024-01-21T22:25:38.276512+0000","last_unstale":"2024-01-21T22:52:23.449441+0000","last_undegraded":"2024-01-21T22:52:23.449441+0000","last_fullsized":"2024-01-21T22:52:23.449441+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:12.153042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":119,"num_write_kb":11355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"25'627","reported_seq":655,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:03.142512+0000","last_change":"2024-01-21T22:25:03.708366+0000","last_active":"2024-01-21T22:52:03.142512+0000","last_peered":"2024-01-21T22:52:03.142512+0000","last_clean":"2024-01-21T22:52:03.142512+0000","last_became_active":"2024-01-21T22:25:03.708101+0000","last_became_peered":"2024-01-21T22:25:03.708101+0000","last_unstale":"2024-01-21T22:52:03.142512+0000","last_undegraded":"2024-01-21T22:52:03.142512+0000","last_fullsized":"2024-01-21T22:52:03.142512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:57:42.881371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":625,"num_write_kb":4435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"25'192","reported_seq":226,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.937318+0000","last_change":"2024-01-21T22:25:04.877993+0000","last_active":"2024-01-21T22:52:19.937318+0000","last_peered":"2024-01-21T22:52:19.937318+0000","last_clean":"2024-01-21T22:52:19.937318+0000","last_became_active":"2024-01-21T22:25:04.877661+0000","last_became_peered":"2024-01-21T22:25:04.877661+0000","last_unstale":"2024-01-21T22:52:19.937318+0000","last_undegraded":"2024-01-21T22:52:19.937318+0000","last_fullsized":"2024-01-21T22:52:19.937318+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:20:40.947205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":145,"num_write_kb":7036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"25'2","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.462672+0000","last_change":"2024-01-21T22:25:03.721559+0000","last_active":"2024-01-21T22:52:28.462672+0000","last_peered":"2024-01-21T22:52:28.462672+0000","last_clean":"2024-01-21T22:52:28.462672+0000","last_became_active":"2024-01-21T22:25:03.721281+0000","last_became_peered":"2024-01-21T22:25:03.721281+0000","last_unstale":"2024-01-21T22:52:28.462672+0000","last_undegraded":"2024-01-21T22:52:28.462672+0000","last_fullsized":"2024-01-21T22:52:28.462672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:55:19.679388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"25'166","reported_seq":210,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.255338+0000","last_change":"2024-01-21T22:25:04.883087+0000","last_active":"2024-01-21T22:52:19.255338+0000","last_peered":"2024-01-21T22:52:19.255338+0000","last_clean":"2024-01-21T22:52:19.255338+0000","last_became_active":"2024-01-21T22:25:04.880227+0000","last_became_peered":"2024-01-21T22:25:04.880227+0000","last_unstale":"2024-01-21T22:52:19.255338+0000","last_undegraded":"2024-01-21T22:52:19.255338+0000","last_fullsized":"2024-01-21T22:52:19.255338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:58:09.653538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":4318,"num_write":118,"num_write_kb":4353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":181,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:48:53.349412+0000","last_change":"2024-01-21T22:24:52.968324+0000","last_active":"2024-01-21T22:48:53.349412+0000","last_peered":"2024-01-21T22:48:53.349412+0000","last_clean":"2024-01-21T22:48:53.349412+0000","last_became_active":"2024-01-21T22:24:52.967018+0000","last_became_peered":"2024-01-21T22:24:52.967018+0000","last_unstale":"2024-01-21T22:48:53.349412+0000","last_undegraded":"2024-01-21T22:48:53.349412+0000","last_fullsized":"2024-01-21T22:48:53.349412+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:24:50.312540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:24:50.312540+0000","last_clean_scrub_stamp":"2024-01-21T22:24:50.312540+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:52:04.800073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"25'4","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.461845+0000","last_change":"2024-01-21T22:25:03.707407+0000","last_active":"2024-01-21T22:52:33.461845+0000","last_peered":"2024-01-21T22:52:33.461845+0000","last_clean":"2024-01-21T22:52:33.461845+0000","last_became_active":"2024-01-21T22:25:03.707085+0000","last_became_peered":"2024-01-21T22:25:03.707085+0000","last_unstale":"2024-01-21T22:52:33.461845+0000","last_undegraded":"2024-01-21T22:52:33.461845+0000","last_fullsized":"2024-01-21T22:52:33.461845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:44:22.501512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"25'169","reported_seq":207,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.558341+0000","last_change":"2024-01-21T22:25:04.868564+0000","last_active":"2024-01-21T22:52:19.558341+0000","last_peered":"2024-01-21T22:52:19.558341+0000","last_clean":"2024-01-21T22:52:19.558341+0000","last_became_active":"2024-01-21T22:25:04.868247+0000","last_became_peered":"2024-01-21T22:25:04.868247+0000","last_unstale":"2024-01-21T22:52:19.558341+0000","last_undegraded":"2024-01-21T22:52:19.558341+0000","last_fullsized":"2024-01-21T22:52:19.558341+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:30:50.039501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2540,"num_write":132,"num_write_kb":4885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"25'1292","reported_seq":1321,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.530564+0000","last_change":"2024-01-21T22:25:03.706070+0000","last_active":"2024-01-21T22:52:33.530564+0000","last_peered":"2024-01-21T22:52:33.530564+0000","last_clean":"2024-01-21T22:52:33.530564+0000","last_became_active":"2024-01-21T22:25:03.705785+0000","last_became_peered":"2024-01-21T22:25:03.705785+0000","last_unstale":"2024-01-21T22:52:33.530564+0000","last_undegraded":"2024-01-21T22:52:33.530564+0000","last_fullsized":"2024-01-21T22:52:33.530564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1292,"log_dups_size":0,"ondisk_log_size":1292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:30.929444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1286,"num_write_kb":8869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"25'183","reported_seq":223,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.224605+0000","last_change":"2024-01-21T22:25:04.845436+0000","last_active":"2024-01-21T22:52:20.224605+0000","last_peered":"2024-01-21T22:52:20.224605+0000","last_clean":"2024-01-21T22:52:20.224605+0000","last_became_active":"2024-01-21T22:25:04.845274+0000","last_became_peered":"2024-01-21T22:25:04.845274+0000","last_unstale":"2024-01-21T22:52:20.224605+0000","last_undegraded":"2024-01-21T22:52:20.224605+0000","last_fullsized":"2024-01-21T22:52:20.224605+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:08:36.686787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":151,"num_write_kb":10622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"25'754","reported_seq":1527,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.492860+0000","last_change":"2024-01-21T22:25:03.709174+0000","last_active":"2024-01-21T22:52:33.492860+0000","last_peered":"2024-01-21T22:52:33.492860+0000","last_clean":"2024-01-21T22:52:33.492860+0000","last_became_active":"2024-01-21T22:25:03.708894+0000","last_became_peered":"2024-01-21T22:25:03.708894+0000","last_unstale":"2024-01-21T22:52:33.492860+0000","last_undegraded":"2024-01-21T22:52:33.492860+0000","last_fullsized":"2024-01-21T22:52:33.492860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:19:26.591140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":608727,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":744,"num_read_kb":1113,"num_write":748,"num_write_kb":1116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"25'180","reported_seq":222,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.222007+0000","last_change":"2024-01-21T22:25:04.711947+0000","last_active":"2024-01-21T22:52:20.222007+0000","last_peered":"2024-01-21T22:52:20.222007+0000","last_clean":"2024-01-21T22:52:20.222007+0000","last_became_active":"2024-01-21T22:25:04.711694+0000","last_became_peered":"2024-01-21T22:25:04.711694+0000","last_unstale":"2024-01-21T22:52:20.222007+0000","last_undegraded":"2024-01-21T22:52:20.222007+0000","last_fullsized":"2024-01-21T22:52:20.222007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:37:57.817795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3556,"num_write":135,"num_write_kb":5944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"25'1294","reported_seq":1323,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:08.470169+0000","last_change":"2024-01-21T22:25:03.706355+0000","last_active":"2024-01-21T22:52:08.470169+0000","last_peered":"2024-01-21T22:52:08.470169+0000","last_clean":"2024-01-21T22:52:08.470169+0000","last_became_active":"2024-01-21T22:25:03.703876+0000","last_became_peered":"2024-01-21T22:25:03.703876+0000","last_unstale":"2024-01-21T22:52:08.470169+0000","last_undegraded":"2024-01-21T22:52:08.470169+0000","last_fullsized":"2024-01-21T22:52:08.470169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1294,"log_dups_size":0,"ondisk_log_size":1294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:42:38.729006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1291,"num_write_kb":8882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"25'223","reported_seq":260,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.222078+0000","last_change":"2024-01-21T22:25:04.885997+0000","last_active":"2024-01-21T22:52:20.222078+0000","last_peered":"2024-01-21T22:52:20.222078+0000","last_clean":"2024-01-21T22:52:20.222078+0000","last_became_active":"2024-01-21T22:25:04.884162+0000","last_became_peered":"2024-01-21T22:25:04.884162+0000","last_unstale":"2024-01-21T22:52:20.222078+0000","last_undegraded":"2024-01-21T22:52:20.222078+0000","last_fullsized":"2024-01-21T22:52:20.222078+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":223,"log_dups_size":0,"ondisk_log_size":223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:44:55.158990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":180,"num_write_kb":15842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"25'3","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.542219+0000","last_change":"2024-01-21T22:25:03.708300+0000","last_active":"2024-01-21T22:52:33.542219+0000","last_peered":"2024-01-21T22:52:33.542219+0000","last_clean":"2024-01-21T22:52:33.542219+0000","last_became_active":"2024-01-21T22:25:03.708036+0000","last_became_peered":"2024-01-21T22:25:03.708036+0000","last_unstale":"2024-01-21T22:52:33.542219+0000","last_undegraded":"2024-01-21T22:52:33.542219+0000","last_fullsized":"2024-01-21T22:52:33.542219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:23:51.196159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"25'184","reported_seq":228,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.941920+0000","last_change":"2024-01-21T22:25:04.844713+0000","last_active":"2024-01-21T22:52:19.941920+0000","last_peered":"2024-01-21T22:52:19.941920+0000","last_clean":"2024-01-21T22:52:19.941920+0000","last_became_active":"2024-01-21T22:25:04.844008+0000","last_became_peered":"2024-01-21T22:25:04.844008+0000","last_unstale":"2024-01-21T22:52:19.941920+0000","last_undegraded":"2024-01-21T22:52:19.941920+0000","last_fullsized":"2024-01-21T22:52:19.941920+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:33:25.760064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":4064,"num_write":142,"num_write_kb":4003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"25'2","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.495347+0000","last_change":"2024-01-21T22:25:03.699093+0000","last_active":"2024-01-21T22:52:28.495347+0000","last_peered":"2024-01-21T22:52:28.495347+0000","last_clean":"2024-01-21T22:52:28.495347+0000","last_became_active":"2024-01-21T22:25:03.698443+0000","last_became_peered":"2024-01-21T22:25:03.698443+0000","last_unstale":"2024-01-21T22:52:28.495347+0000","last_undegraded":"2024-01-21T22:52:28.495347+0000","last_fullsized":"2024-01-21T22:52:28.495347+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:20:39.699087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"25'204","reported_seq":239,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.126809+0000","last_change":"2024-01-21T22:25:04.842733+0000","last_active":"2024-01-21T22:52:20.126809+0000","last_peered":"2024-01-21T22:52:20.126809+0000","last_clean":"2024-01-21T22:52:20.126809+0000","last_became_active":"2024-01-21T22:25:04.842479+0000","last_became_peered":"2024-01-21T22:25:04.842479+0000","last_unstale":"2024-01-21T22:52:20.126809+0000","last_undegraded":"2024-01-21T22:52:20.126809+0000","last_fullsized":"2024-01-21T22:52:20.126809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:51:06.931790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":156,"num_write_kb":5205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"25'1417","reported_seq":1445,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:36.159123+0000","last_change":"2024-01-21T22:25:03.720306+0000","last_active":"2024-01-21T22:52:36.159123+0000","last_peered":"2024-01-21T22:52:36.159123+0000","last_clean":"2024-01-21T22:52:36.159123+0000","last_became_active":"2024-01-21T22:25:03.720067+0000","last_became_peered":"2024-01-21T22:25:03.720067+0000","last_unstale":"2024-01-21T22:52:36.159123+0000","last_undegraded":"2024-01-21T22:52:36.159123+0000","last_fullsized":"2024-01-21T22:52:36.159123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1417,"log_dups_size":0,"ondisk_log_size":1417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:52:22.985590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3483,"num_write_kb":1511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"25'220","reported_seq":270,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.252712+0000","last_change":"2024-01-21T22:25:04.859628+0000","last_active":"2024-01-21T22:52:20.252712+0000","last_peered":"2024-01-21T22:52:20.252712+0000","last_clean":"2024-01-21T22:52:20.252712+0000","last_became_active":"2024-01-21T22:25:04.859270+0000","last_became_peered":"2024-01-21T22:25:04.859270+0000","last_unstale":"2024-01-21T22:52:20.252712+0000","last_undegraded":"2024-01-21T22:52:20.252712+0000","last_fullsized":"2024-01-21T22:52:20.252712+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:56:24.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":5842,"num_write":177,"num_write_kb":7531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"25'6","reported_seq":34,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.531715+0000","last_change":"2024-01-21T22:25:03.722006+0000","last_active":"2024-01-21T22:52:33.531715+0000","last_peered":"2024-01-21T22:52:33.531715+0000","last_clean":"2024-01-21T22:52:33.531715+0000","last_became_active":"2024-01-21T22:25:03.721275+0000","last_became_peered":"2024-01-21T22:25:03.721275+0000","last_unstale":"2024-01-21T22:52:33.531715+0000","last_undegraded":"2024-01-21T22:52:33.531715+0000","last_fullsized":"2024-01-21T22:52:33.531715+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:43:53.801587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"25'175","reported_seq":215,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.145841+0000","last_change":"2024-01-21T22:25:04.858864+0000","last_active":"2024-01-21T22:52:20.145841+0000","last_peered":"2024-01-21T22:52:20.145841+0000","last_clean":"2024-01-21T22:52:20.145841+0000","last_became_active":"2024-01-21T22:25:04.858586+0000","last_became_peered":"2024-01-21T22:25:04.858586+0000","last_unstale":"2024-01-21T22:52:20.145841+0000","last_undegraded":"2024-01-21T22:52:20.145841+0000","last_fullsized":"2024-01-21T22:52:20.145841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:32.177562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":132,"num_write_kb":7745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"25'670","reported_seq":698,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.532991+0000","last_change":"2024-01-21T22:25:03.718436+0000","last_active":"2024-01-21T22:52:33.532991+0000","last_peered":"2024-01-21T22:52:33.532991+0000","last_clean":"2024-01-21T22:52:33.532991+0000","last_became_active":"2024-01-21T22:25:03.718171+0000","last_became_peered":"2024-01-21T22:25:03.718171+0000","last_unstale":"2024-01-21T22:52:33.532991+0000","last_undegraded":"2024-01-21T22:52:33.532991+0000","last_fullsized":"2024-01-21T22:52:33.532991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":670,"log_dups_size":0,"ondisk_log_size":670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:05:40.702113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":666,"num_write_kb":4450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"25'191","reported_seq":227,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:07.603453+0000","last_change":"2024-01-21T22:25:04.875736+0000","last_active":"2024-01-21T22:52:07.603453+0000","last_peered":"2024-01-21T22:52:07.603453+0000","last_clean":"2024-01-21T22:52:07.603453+0000","last_became_active":"2024-01-21T22:25:04.875368+0000","last_became_peered":"2024-01-21T22:25:04.875368+0000","last_unstale":"2024-01-21T22:52:07.603453+0000","last_undegraded":"2024-01-21T22:52:07.603453+0000","last_fullsized":"2024-01-21T22:52:07.603453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":191,"log_dups_size":0,"ondisk_log_size":191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:39:31.273250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":138,"num_write_kb":6957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"25'3","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.491094+0000","last_change":"2024-01-21T22:25:03.722820+0000","last_active":"2024-01-21T22:52:28.491094+0000","last_peered":"2024-01-21T22:52:28.491094+0000","last_clean":"2024-01-21T22:52:28.491094+0000","last_became_active":"2024-01-21T22:25:03.722525+0000","last_became_peered":"2024-01-21T22:25:03.722525+0000","last_unstale":"2024-01-21T22:52:28.491094+0000","last_undegraded":"2024-01-21T22:52:28.491094+0000","last_fullsized":"2024-01-21T22:52:28.491094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:40:04.306585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"25'169","reported_seq":209,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.378058+0000","last_change":"2024-01-21T22:25:04.857451+0000","last_active":"2024-01-21T22:52:23.378058+0000","last_peered":"2024-01-21T22:52:23.378058+0000","last_clean":"2024-01-21T22:52:23.378058+0000","last_became_active":"2024-01-21T22:25:04.857205+0000","last_became_peered":"2024-01-21T22:25:04.857205+0000","last_unstale":"2024-01-21T22:52:23.378058+0000","last_undegraded":"2024-01-21T22:52:23.378058+0000","last_fullsized":"2024-01-21T22:52:23.378058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:12.153042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":126,"num_write_kb":4840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"25'2","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:51:08.155593+0000","last_change":"2024-01-21T22:25:03.708574+0000","last_active":"2024-01-21T22:51:08.155593+0000","last_peered":"2024-01-21T22:51:08.155593+0000","last_clean":"2024-01-21T22:51:08.155593+0000","last_became_active":"2024-01-21T22:25:03.708318+0000","last_became_peered":"2024-01-21T22:25:03.708318+0000","last_unstale":"2024-01-21T22:51:08.155593+0000","last_undegraded":"2024-01-21T22:51:08.155593+0000","last_fullsized":"2024-01-21T22:51:08.155593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:50:40.058463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"25'214","reported_seq":274,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.129443+0000","last_change":"2024-01-21T22:25:04.843079+0000","last_active":"2024-01-21T22:52:20.129443+0000","last_peered":"2024-01-21T22:52:20.129443+0000","last_clean":"2024-01-21T22:52:20.129443+0000","last_became_active":"2024-01-21T22:25:04.842815+0000","last_became_peered":"2024-01-21T22:25:04.842815+0000","last_unstale":"2024-01-21T22:52:20.129443+0000","last_undegraded":"2024-01-21T22:52:20.129443+0000","last_fullsized":"2024-01-21T22:52:20.129443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":214,"log_dups_size":0,"ondisk_log_size":214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:16:41.440927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":8382,"num_write":174,"num_write_kb":9170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"25'763","reported_seq":791,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.532623+0000","last_change":"2024-01-21T22:25:03.710506+0000","last_active":"2024-01-21T22:52:33.532623+0000","last_peered":"2024-01-21T22:52:33.532623+0000","last_clean":"2024-01-21T22:52:33.532623+0000","last_became_active":"2024-01-21T22:25:03.710262+0000","last_became_peered":"2024-01-21T22:25:03.710262+0000","last_unstale":"2024-01-21T22:52:33.532623+0000","last_undegraded":"2024-01-21T22:52:33.532623+0000","last_fullsized":"2024-01-21T22:52:33.532623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:25:49.132306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":756,"num_write_kb":4574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"25'188","reported_seq":229,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.941642+0000","last_change":"2024-01-21T22:25:04.876469+0000","last_active":"2024-01-21T22:52:19.941642+0000","last_peered":"2024-01-21T22:52:19.941642+0000","last_clean":"2024-01-21T22:52:19.941642+0000","last_became_active":"2024-01-21T22:25:04.875619+0000","last_became_peered":"2024-01-21T22:25:04.875619+0000","last_unstale":"2024-01-21T22:52:19.941642+0000","last_undegraded":"2024-01-21T22:52:19.941642+0000","last_fullsized":"2024-01-21T22:52:19.941642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:30:20.167607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3556,"num_write":140,"num_write_kb":4746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"25'140","reported_seq":173,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:20.144836+0000","last_change":"2024-01-21T22:25:04.891067+0000","last_active":"2024-01-21T22:52:20.144836+0000","last_peered":"2024-01-21T22:52:20.144836+0000","last_clean":"2024-01-21T22:52:20.144836+0000","last_became_active":"2024-01-21T22:25:04.890515+0000","last_became_peered":"2024-01-21T22:25:04.890515+0000","last_unstale":"2024-01-21T22:52:20.144836+0000","last_undegraded":"2024-01-21T22:52:20.144836+0000","last_fullsized":"2024-01-21T22:52:20.144836+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:25:48.726591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":88,"num_write_kb":3797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"25'196","reported_seq":223,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:35.717345+0000","last_change":"2024-01-21T22:25:03.719412+0000","last_active":"2024-01-21T22:52:35.717345+0000","last_peered":"2024-01-21T22:52:35.717345+0000","last_clean":"2024-01-21T22:52:35.717345+0000","last_became_active":"2024-01-21T22:25:03.718163+0000","last_became_peered":"2024-01-21T22:25:03.718163+0000","last_unstale":"2024-01-21T22:52:35.717345+0000","last_undegraded":"2024-01-21T22:52:35.717345+0000","last_fullsized":"2024-01-21T22:52:35.717345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:56:22.817420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1895390,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":195,"num_write_kb":1969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"25'203","reported_seq":235,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.379097+0000","last_change":"2024-01-21T22:25:04.889166+0000","last_active":"2024-01-21T22:52:23.379097+0000","last_peered":"2024-01-21T22:52:23.379097+0000","last_clean":"2024-01-21T22:52:23.379097+0000","last_became_active":"2024-01-21T22:25:04.888910+0000","last_became_peered":"2024-01-21T22:25:04.888910+0000","last_unstale":"2024-01-21T22:52:23.379097+0000","last_undegraded":"2024-01-21T22:52:23.379097+0000","last_fullsized":"2024-01-21T22:52:23.379097+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:55:53.892439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":155,"num_write_kb":13062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"25'1484","reported_seq":1511,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.469678+0000","last_change":"2024-01-21T22:25:03.718932+0000","last_active":"2024-01-21T22:52:33.469678+0000","last_peered":"2024-01-21T22:52:33.469678+0000","last_clean":"2024-01-21T22:52:33.469678+0000","last_became_active":"2024-01-21T22:25:03.718649+0000","last_became_peered":"2024-01-21T22:25:03.718649+0000","last_unstale":"2024-01-21T22:52:33.469678+0000","last_undegraded":"2024-01-21T22:52:33.469678+0000","last_fullsized":"2024-01-21T22:52:33.469678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1484,"log_dups_size":0,"ondisk_log_size":1484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:30.929444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1478,"num_write_kb":9119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"25'204","reported_seq":259,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.264524+0000","last_change":"2024-01-21T22:25:04.715429+0000","last_active":"2024-01-21T22:52:19.264524+0000","last_peered":"2024-01-21T22:52:19.264524+0000","last_clean":"2024-01-21T22:52:19.264524+0000","last_became_active":"2024-01-21T22:25:04.714720+0000","last_became_peered":"2024-01-21T22:25:04.714720+0000","last_unstale":"2024-01-21T22:52:19.264524+0000","last_undegraded":"2024-01-21T22:52:19.264524+0000","last_fullsized":"2024-01-21T22:52:19.264524+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:44:23.749631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":6858,"num_write":147,"num_write_kb":17332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"25'747","reported_seq":776,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.457894+0000","last_change":"2024-01-21T22:25:03.703289+0000","last_active":"2024-01-21T22:52:28.457894+0000","last_peered":"2024-01-21T22:52:28.457894+0000","last_clean":"2024-01-21T22:52:28.457894+0000","last_became_active":"2024-01-21T22:25:03.703020+0000","last_became_peered":"2024-01-21T22:25:03.703020+0000","last_unstale":"2024-01-21T22:52:28.457894+0000","last_undegraded":"2024-01-21T22:52:28.457894+0000","last_fullsized":"2024-01-21T22:52:28.457894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:10.904923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":743,"num_write_kb":4567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"25'211","reported_seq":261,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.392685+0000","last_change":"2024-01-21T22:25:04.715894+0000","last_active":"2024-01-21T22:52:23.392685+0000","last_peered":"2024-01-21T22:52:23.392685+0000","last_clean":"2024-01-21T22:52:23.392685+0000","last_became_active":"2024-01-21T22:25:04.715652+0000","last_became_peered":"2024-01-21T22:25:04.715652+0000","last_unstale":"2024-01-21T22:52:23.392685+0000","last_undegraded":"2024-01-21T22:52:23.392685+0000","last_fullsized":"2024-01-21T22:52:23.392685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:13:04.601911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":5588,"num_write":182,"num_write_kb":13243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"25'762","reported_seq":791,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.498304+0000","last_change":"2024-01-21T22:25:03.702339+0000","last_active":"2024-01-21T22:52:23.498304+0000","last_peered":"2024-01-21T22:52:23.498304+0000","last_clean":"2024-01-21T22:52:23.498304+0000","last_became_active":"2024-01-21T22:25:03.702087+0000","last_became_peered":"2024-01-21T22:25:03.702087+0000","last_unstale":"2024-01-21T22:52:23.498304+0000","last_undegraded":"2024-01-21T22:52:23.498304+0000","last_fullsized":"2024-01-21T22:52:23.498304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":762,"log_dups_size":0,"ondisk_log_size":762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:51:05.683671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":760,"num_write_kb":4560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":833,"num_read_kb":211582,"num_write":8969,"num_write_kb":506711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11784,"ondisk_log_size":11784,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":82197363,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":744,"num_read_kb":1113,"num_write":17679,"num_write_kb":90247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164454400,"data_stored":164394726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15719,"ondisk_log_size":15719,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542483,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62372,"kb_used_data":13148,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94309468,"statfs":{"total":96636764160,"available":96572895232,"internally_reserved":0,"allocated":13463552,"data_stored":13288704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67700000000000005,"15min":0.68500000000000005},"min":{"1min":0.46000000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89600000000000002,"5min":1.0189999999999999,"15min":1.0640000000000001},"last":0.496},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70699999999999996,"15min":0.69399999999999995},"min":{"1min":0.497,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.081,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.376}]},{"osd":1,"last update":"Sun Jan 21 22:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71599999999999997,"15min":0.71099999999999997},"min":{"1min":0.55000000000000004,"5min":0.51700000000000002,"15min":0.34000000000000002},"max":{"1min":0.96899999999999997,"5min":1.159,"15min":1.4179999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.73399999999999999,"15min":0.72299999999999998},"min":{"1min":0.56000000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95599999999999996,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.42999999999999999}]},{"osd":2,"last update":"Sun Jan 21 22:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69299999999999995,"15min":0.70099999999999996},"min":{"1min":0.49399999999999999,"5min":0.46000000000000002,"15min":0.39300000000000002},"max":{"1min":0.94699999999999995,"5min":1.0069999999999999,"15min":1.052},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73899999999999999,"15min":0.72499999999999998},"min":{"1min":0.58099999999999996,"5min":0.438,"15min":0.38},"max":{"1min":1.02,"5min":1.1140000000000001,"15min":1.1799999999999999},"last":0.55000000000000004}]},{"osd":3,"last update":"Sun Jan 21 22:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71099999999999997,"15min":0.71399999999999997},"min":{"1min":0.54500000000000004,"5min":0.42699999999999999,"15min":0.42099999999999999},"max":{"1min":1.048,"5min":1.093,"15min":1.1539999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.73199999999999998,"15min":0.74099999999999999},"min":{"1min":0.56299999999999994,"5min":0.48399999999999999,"15min":0.45600000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.181},"last":0.46100000000000002}]},{"osd":4,"last update":"Sun Jan 21 22:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.35099999999999998,"15min":0.34499999999999997},"min":{"1min":0.26600000000000001,"5min":0.245,"15min":0.22700000000000001},"max":{"1min":0.39800000000000002,"5min":0.57099999999999995,"15min":0.58299999999999996},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.35199999999999998,"15min":0.34699999999999998},"min":{"1min":0.27300000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.41699999999999998,"5min":0.66700000000000004,"15min":0.68100000000000005},"last":0.44800000000000001}]},{"osd":5,"last update":"Sun Jan 21 22:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34300000000000003,"15min":0.34200000000000003},"min":{"1min":0.255,"5min":0.255,"15min":0.249},"max":{"1min":0.44500000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38100000000000001,"15min":0.372},"min":{"1min":0.31,"5min":0.26700000000000002,"15min":0.249},"max":{"1min":0.48599999999999999,"5min":0.54100000000000004,"15min":0.624},"last":0.39200000000000002}]},{"osd":6,"last update":"Sun Jan 21 22:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.40000000000000002,"15min":0.39500000000000002},"min":{"1min":0.33400000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.45200000000000001,"5min":0.57199999999999995,"15min":0.67300000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.313,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.52300000000000002,"5min":0.60699999999999998,"15min":0.63700000000000001},"last":0.41099999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542483,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43720,"kb_used_data":12544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328120,"statfs":{"total":96636764160,"available":96591994880,"internally_reserved":0,"allocated":12845056,"data_stored":12679853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69299999999999995,"15min":0.70099999999999996},"min":{"1min":0.497,"5min":0.48299999999999998,"15min":0.45600000000000002},"max":{"1min":0.89200000000000002,"5min":1.0189999999999999,"15min":3.3380000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.54100000000000004,"5min":0.495,"15min":0.44600000000000001},"max":{"1min":0.93799999999999994,"5min":1.038,"15min":1.0669999999999999},"last":0.92300000000000004}]},{"osd":1,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72099999999999997,"15min":0.70699999999999996},"min":{"1min":0.46500000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0529999999999999,"5min":1.0860000000000001,"15min":2.9860000000000002},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72299999999999998,"15min":0.72099999999999997},"min":{"1min":0.52900000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.1020000000000001},"last":0.57999999999999996}]},{"osd":2,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.49299999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.88200000000000001,"5min":1.1220000000000001,"15min":7.5439999999999996},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71999999999999997,"15min":0.73199999999999998},"min":{"1min":0.59499999999999997,"5min":0.56200000000000006,"15min":0.495},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.81399999999999995}]},{"osd":3,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72399999999999998,"15min":0.70499999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.42599999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73099999999999998,"15min":0.78100000000000003},"min":{"1min":0.56399999999999995,"5min":0.52700000000000002,"15min":0.39400000000000002},"max":{"1min":0.90100000000000002,"5min":1.075,"15min":8.4689999999999994},"last":0.73199999999999998}]},{"osd":4,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34300000000000003,"15min":0.33800000000000002},"min":{"1min":0.26000000000000001,"5min":0.245,"15min":0.222},"max":{"1min":0.45200000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.35599999999999998,"15min":0.35199999999999998},"min":{"1min":0.24199999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.45100000000000001,"5min":0.45600000000000002,"15min":0.91900000000000004},"last":0.32200000000000001}]},{"osd":5,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35199999999999998,"15min":0.35399999999999998},"min":{"1min":0.30199999999999999,"5min":0.26400000000000001,"15min":0.26100000000000001},"max":{"1min":0.40200000000000002,"5min":0.53800000000000003,"15min":0.61299999999999999},"last":0.374},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.377,"15min":0.36799999999999999},"min":{"1min":0.29599999999999999,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.45300000000000001,"5min":0.46800000000000003,"15min":0.52200000000000002},"last":0.30199999999999999}]},{"osd":7,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.38},"min":{"1min":0.29799999999999999,"5min":0.27100000000000002,"15min":0.26600000000000001},"max":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.96099999999999997},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.39200000000000002,"15min":0.39100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28799999999999998,"15min":0.27300000000000002},"max":{"1min":0.45200000000000001,"5min":0.54900000000000004,"15min":0.98699999999999999},"last":0.33800000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542483,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":101608,"kb_used_data":26016,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94270232,"statfs":{"total":96636764160,"available":96532717568,"internally_reserved":0,"allocated":26640384,"data_stored":26461952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.68400000000000005,"15min":0.68899999999999995},"min":{"1min":0.46300000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.499,"5min":0.432,"15min":0.41499999999999998},"max":{"1min":0.83799999999999997,"5min":1.0149999999999999,"15min":1.1020000000000001},"last":0.55100000000000005}]},{"osd":1,"last update":"Sun Jan 21 22:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67200000000000004,"15min":0.68999999999999995},"min":{"1min":0.56599999999999995,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91500000000000004,"5min":0.95599999999999996,"15min":1.1950000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.74099999999999999,"15min":0.72799999999999998},"min":{"1min":0.50800000000000001,"5min":0.49199999999999999,"15min":0.40999999999999998},"max":{"1min":0.96999999999999997,"5min":1.111,"15min":1.151},"last":0.83699999999999997}]},{"osd":2,"last update":"Sun Jan 21 22:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.72099999999999997,"15min":0.71899999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.434},"max":{"1min":1.046,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70399999999999996,"15min":0.71499999999999997},"min":{"1min":0.53300000000000003,"5min":0.437,"15min":0.437},"max":{"1min":0.85399999999999998,"5min":1.143,"15min":1.149},"last":0.70499999999999996}]},{"osd":3,"last update":"Sun Jan 21 22:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.70799999999999996,"15min":0.73399999999999999},"min":{"1min":0.48499999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97499999999999998,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.71799999999999997},"min":{"1min":0.58799999999999997,"5min":0.46200000000000002,"15min":0.373},"max":{"1min":0.97199999999999998,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.81999999999999995}]},{"osd":4,"last update":"Sun Jan 21 22:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.32800000000000001,"15min":0.33400000000000002},"min":{"1min":0.254,"5min":0.23599999999999999,"15min":0.22800000000000001},"max":{"1min":0.433,"5min":0.46300000000000002,"15min":0.47099999999999997},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.34399999999999997,"15min":0.34999999999999998},"min":{"1min":0.28100000000000003,"5min":0.26500000000000001,"15min":0.254},"max":{"1min":0.41199999999999998,"5min":0.50700000000000001,"15min":0.55400000000000005},"last":0.317}]},{"osd":6,"last update":"Sun Jan 21 22:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.27700000000000002,"5min":0.26300000000000001,"15min":0.26000000000000001},"max":{"1min":0.38600000000000001,"5min":0.48899999999999999,"15min":0.51800000000000002},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.25600000000000001},"max":{"1min":0.41699999999999998,"5min":0.48099999999999998,"15min":0.51100000000000001},"last":0.34999999999999998}]},{"osd":7,"last update":"Sun Jan 21 22:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.38,"15min":0.38200000000000001},"min":{"1min":0.28499999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.441,"5min":0.47799999999999998,"15min":0.64000000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.38200000000000001,"15min":0.38400000000000001},"min":{"1min":0.28199999999999997,"5min":0.27600000000000002,"15min":0.27100000000000002},"max":{"1min":0.45300000000000001,"5min":0.54100000000000004,"15min":0.57099999999999995},"last":0.41399999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542483,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65944,"kb_used_data":12560,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94305896,"statfs":{"total":96636764160,"available":96569237504,"internally_reserved":0,"allocated":12861440,"data_stored":12681197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.51100000000000001,"5min":0.46200000000000002,"15min":0.38800000000000001},"max":{"1min":0.97299999999999998,"5min":1.05,"15min":1.0880000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69399999999999995,"15min":0.71499999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.41099999999999998},"max":{"1min":0.998,"5min":1.008,"15min":2.9649999999999999},"last":0.93400000000000005}]},{"osd":1,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.70599999999999996,"15min":0.71899999999999997},"min":{"1min":0.45800000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.9410000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.70199999999999996,"15min":0.71399999999999997},"min":{"1min":0.44,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.615},"last":0.67100000000000004}]},{"osd":2,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70499999999999996,"15min":0.71499999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.95799999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73499999999999999,"15min":0.71799999999999997},"min":{"1min":0.54700000000000004,"5min":0.505,"15min":0.40400000000000003},"max":{"1min":1.05,"5min":1.05,"15min":1.4710000000000001},"last":0.70099999999999996}]},{"osd":3,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69899999999999995,"15min":0.69799999999999995},"min":{"1min":0.55400000000000005,"5min":0.48499999999999999,"15min":0.43099999999999999},"max":{"1min":0.97499999999999998,"5min":1.0640000000000001,"15min":1.0720000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.72699999999999998,"15min":0.745},"min":{"1min":0.53800000000000003,"5min":0.52300000000000002,"15min":0.50900000000000001},"max":{"1min":0.92700000000000005,"5min":1.0600000000000001,"15min":1.1160000000000001},"last":0.68600000000000005}]},{"osd":5,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34499999999999997,"15min":0.34200000000000003},"min":{"1min":0.251,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.432,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.35099999999999998,"15min":0.34799999999999998},"min":{"1min":0.28000000000000003,"5min":0.252,"15min":0.249},"max":{"1min":0.42299999999999999,"5min":0.443,"15min":0.49399999999999999},"last":0.45300000000000001}]},{"osd":6,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34999999999999998,"15min":0.34699999999999998},"min":{"1min":0.28100000000000003,"5min":0.24299999999999999,"15min":0.23599999999999999},"max":{"1min":0.42099999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.376,"5min":0.378,"15min":0.374},"min":{"1min":0.29199999999999998,"5min":0.27700000000000002,"15min":0.26400000000000001},"max":{"1min":0.46800000000000003,"5min":0.51500000000000001,"15min":0.54000000000000004},"last":0.43099999999999999}]},{"osd":7,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.27600000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.52000000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39600000000000002,"15min":0.39000000000000001},"min":{"1min":0.317,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.47699999999999998,"5min":0.626,"15min":0.626},"last":0.52900000000000003}]}]},{"osd":3,"up_from":14,"seq":60129542484,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104396,"kb_used_data":37124,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94267444,"statfs":{"total":96636764160,"available":96529862656,"internally_reserved":0,"allocated":38014976,"data_stored":37846259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:51:40 2024","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.069,"15min":1.089},"min":{"1min":0.73999999999999999,"5min":0.69699999999999995,"15min":0.56499999999999995},"max":{"1min":1.8169999999999999,"5min":3.9849999999999999,"15min":6.1879999999999997},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.274,"15min":1.2769999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.67300000000000004},"max":{"1min":1.53,"5min":8.6639999999999997,"15min":8.6639999999999997},"last":1.04}]},{"osd":1,"last update":"Sun Jan 21 22:51:40 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.2889999999999999,"15min":1.2709999999999999},"min":{"1min":0.751,"5min":0.72999999999999998,"15min":0.67000000000000004},"max":{"1min":1.7010000000000001,"5min":17.492999999999999,"15min":17.492999999999999},"last":1.401},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.4379999999999999,"15min":1.423},"min":{"1min":1.05,"5min":1.0269999999999999,"15min":0.874},"max":{"1min":1.635,"5min":8.9629999999999992,"15min":8.9629999999999992},"last":1.1850000000000001}]},{"osd":2,"last update":"Sun Jan 21 22:51:40 2024","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.401,"15min":1.3740000000000001},"min":{"1min":1.115,"5min":0.86199999999999999,"15min":0.86199999999999999},"max":{"1min":1.585,"5min":8.5530000000000008,"15min":8.5530000000000008},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.462,"5min":1.5680000000000001,"15min":1.5169999999999999},"min":{"1min":1.3240000000000001,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.639,"5min":17.664999999999999,"15min":17.664999999999999},"last":1.304}]},{"osd":4,"last update":"Sun Jan 21 22:51:40 2024","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.169,"15min":1.151},"min":{"1min":0.82299999999999995,"5min":0.78000000000000003,"15min":0.72099999999999997},"max":{"1min":1.5580000000000001,"5min":4.5099999999999998,"15min":4.9080000000000004},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.054,"5min":1.1659999999999999,"15min":1.151},"min":{"1min":0.83499999999999996,"5min":0.77800000000000002,"15min":0.71799999999999997},"max":{"1min":1.304,"5min":5.6559999999999997,"15min":5.6559999999999997},"last":0.81100000000000005}]},{"osd":5,"last update":"Sun Jan 21 22:51:40 2024","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.173,"15min":1.1950000000000001},"min":{"1min":0.93600000000000005,"5min":0.77000000000000002,"15min":0.754},"max":{"1min":1.472,"5min":3.2040000000000002,"15min":9.2370000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.353,"15min":1.3100000000000001},"min":{"1min":0.871,"5min":0.80600000000000005,"15min":0.72399999999999998},"max":{"1min":1.9330000000000001,"5min":8.7420000000000009,"15min":8.7420000000000009},"last":1.089}]},{"osd":6,"last update":"Sun Jan 21 22:51:40 2024","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3420000000000001,"15min":1.3600000000000001},"min":{"1min":0.88200000000000001,"5min":0.874,"15min":0.85899999999999999},"max":{"1min":1.6839999999999999,"5min":3.3199999999999998,"15min":9.3719999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.3939999999999999,"15min":1.379},"min":{"1min":0.95899999999999996,"5min":0.89300000000000002,"15min":0.78700000000000003},"max":{"1min":1.7949999999999999,"5min":5.806,"15min":5.806},"last":1.224}]},{"osd":7,"last update":"Sun Jan 21 22:51:40 2024","interfaces":[{"interface":"back","average":{"1min":1.49,"5min":1.476,"15min":1.492},"min":{"1min":1.109,"5min":0.93700000000000006,"15min":0.86099999999999999},"max":{"1min":1.9019999999999999,"5min":3.27,"15min":9.4610000000000003},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.526,"5min":1.5629999999999999,"15min":1.522},"min":{"1min":1.1060000000000001,"5min":1.0580000000000001,"15min":0.97399999999999998},"max":{"1min":2.0339999999999998,"5min":8.8719999999999999,"15min":8.8719999999999999},"last":1.266}]}]},{"osd":2,"up_from":14,"seq":60129542482,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61776,"kb_used_data":12552,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94310064,"statfs":{"total":96636764160,"available":96573505536,"internally_reserved":0,"allocated":12853248,"data_stored":12680525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0509999999999999,"15min":1.04},"min":{"1min":0.79100000000000004,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5369999999999999,"5min":1.9410000000000001,"15min":2.0419999999999998},"last":1.294},{"interface":"front","average":{"1min":1.349,"5min":1.306,"15min":1.2769999999999999},"min":{"1min":0.95599999999999996,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.6299999999999999,"5min":2.2770000000000001,"15min":2.2770000000000001},"last":1.484}]},{"osd":1,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.29,"15min":1.2629999999999999},"min":{"1min":0.88400000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.605,"5min":1.9399999999999999,"15min":2.234},"last":1.431},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.3540000000000001,"15min":1.3759999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.8540000000000001,"5min":2.411,"15min":2.411},"last":1.149}]},{"osd":3,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.27,"15min":1.296},"min":{"1min":0.96799999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.556,"5min":1.788,"15min":1.9690000000000001},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.429,"15min":1.4410000000000001},"min":{"1min":1.2230000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.681,"5min":2.0659999999999998,"15min":2.1320000000000001},"last":1.54}]},{"osd":4,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.149,"15min":1.155},"min":{"1min":0.90100000000000002,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.345,"5min":1.776,"15min":2.0259999999999998},"last":1.615},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0720000000000001,"15min":1.0820000000000001},"min":{"1min":0.69999999999999996,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4199999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.88600000000000001}]},{"osd":5,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.79900000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.415,"5min":1.623,"15min":1.669},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.244,"5min":1.2809999999999999,"15min":1.29},"min":{"1min":1.0860000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5720000000000001,"5min":1.8859999999999999,"15min":2.125},"last":1.6639999999999999}]},{"osd":6,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.244,"15min":1.25},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.6160000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.2130000000000001,"15min":1.254},"min":{"1min":0.96799999999999997,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.5880000000000001,"5min":1.8600000000000001,"15min":1.8919999999999999},"last":1.0249999999999999}]},{"osd":7,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.377,"15min":1.4019999999999999},"min":{"1min":1.034,"5min":0.748,"15min":0.748},"max":{"1min":1.7450000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3640000000000001,"15min":1.391},"min":{"1min":1.077,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.694,"5min":1.9139999999999999,"15min":2.004},"last":1.2350000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542483,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76556,"kb_used_data":23172,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94295284,"statfs":{"total":96636764160,"available":96558370816,"internally_reserved":0,"allocated":23728128,"data_stored":23554289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.0620000000000001,"15min":1.0509999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.3839999999999999,"5min":2.3839999999999999,"15min":3.613},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.2430000000000001,"15min":1.214},"min":{"1min":0.76000000000000001,"5min":0.72499999999999998,"15min":0.63800000000000001},"max":{"1min":3.3879999999999999,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.94999999999999996}]},{"osd":2,"last update":"Sun Jan 21 22:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.248,"15min":1.236},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.73799999999999999},"max":{"1min":3.056,"5min":3.056,"15min":4.2309999999999999},"last":0.90400000000000003},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.355,"15min":1.353},"min":{"1min":0.996,"5min":0.82499999999999996,"15min":0.76700000000000002},"max":{"1min":2.8159999999999998,"5min":2.8159999999999998,"15min":3.2450000000000001},"last":0.85899999999999999}]},{"osd":3,"last update":"Sun Jan 21 22:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.4339999999999999,"15min":1.425},"min":{"1min":1.0189999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":4.0890000000000004,"5min":4.0890000000000004,"15min":4.0890000000000004},"last":1.218},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.3540000000000001,"15min":1.3560000000000001},"min":{"1min":0.874,"5min":0.84599999999999997,"15min":0.78100000000000003},"max":{"1min":3.8519999999999999,"5min":3.8519999999999999,"15min":3.8519999999999999},"last":1.2689999999999999}]},{"osd":4,"last update":"Sun Jan 21 22:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0660000000000001,"15min":1.028},"min":{"1min":0.67700000000000005,"5min":0.67200000000000004,"15min":0.65600000000000003},"max":{"1min":2.8639999999999999,"5min":2.8639999999999999,"15min":2.8639999999999999},"last":1.006},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.206,"15min":1.1879999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.74299999999999999},"max":{"1min":2.5939999999999999,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":1.3300000000000001}]},{"osd":5,"last update":"Sun Jan 21 22:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1140000000000001,"15min":1.1100000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71099999999999997},"max":{"1min":3.0139999999999998,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":1.143},{"interface":"front","average":{"1min":1.194,"5min":1.26,"15min":1.2410000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.972,"5min":2.972,"15min":2.972},"last":1.099}]},{"osd":6,"last update":"Sun Jan 21 22:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.302,"15min":1.3009999999999999},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.79300000000000004},"max":{"1min":3.121,"5min":3.121,"15min":3.121},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.4279999999999999,"15min":1.399},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.88200000000000001},"max":{"1min":3.3410000000000002,"5min":3.3410000000000002,"15min":3.3410000000000002},"last":1.1899999999999999}]},{"osd":7,"last update":"Sun Jan 21 22:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.86799999999999999},"max":{"1min":2.718,"5min":2.718,"15min":2.718},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4410000000000001,"15min":1.4710000000000001},"min":{"1min":1.002,"5min":1.002,"15min":0.93200000000000005},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":1.4770000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542484,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":89804,"kb_used_data":26692,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94282036,"statfs":{"total":96636764160,"available":96544804864,"internally_reserved":0,"allocated":27332608,"data_stored":27158211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0309999999999999,"15min":1.0640000000000001},"min":{"1min":0.70499999999999996,"5min":0.624,"15min":0.621},"max":{"1min":1.6080000000000001,"5min":1.6870000000000001,"15min":1.988},"last":1.208},{"interface":"front","average":{"1min":1.288,"5min":1.232,"15min":1.2889999999999999},"min":{"1min":0.90100000000000002,"5min":0.70199999999999996,"15min":0.67200000000000004},"max":{"1min":1.609,"5min":1.718,"15min":11.01},"last":0.89600000000000002}]},{"osd":2,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.306,"15min":1.3300000000000001},"min":{"1min":1.2050000000000001,"5min":0.83799999999999997,"15min":0.81699999999999995},"max":{"1min":1.8580000000000001,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.3480000000000001,"15min":1.3879999999999999},"min":{"1min":0.96199999999999997,"5min":0.84499999999999997,"15min":0.84399999999999997},"max":{"1min":1.742,"5min":1.9690000000000001,"15min":11.308999999999999},"last":1.127}]},{"osd":3,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.268,"15min":1.3129999999999999},"min":{"1min":0.98999999999999999,"5min":0.85399999999999998,"15min":0.67000000000000004},"max":{"1min":1.573,"5min":1.671,"15min":9.2929999999999993},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.462,"5min":1.4390000000000001,"15min":1.4850000000000001},"min":{"1min":1.1679999999999999,"5min":1.01,"15min":0.91700000000000004},"max":{"1min":1.696,"5min":2.0390000000000001,"15min":11.4},"last":1.302}]},{"osd":4,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0660000000000001,"15min":1.0860000000000001},"min":{"1min":0.82099999999999995,"5min":0.70999999999999996,"15min":0.66200000000000003},"max":{"1min":1.3200000000000001,"5min":1.593,"15min":2.0110000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.157,"15min":1.2050000000000001},"min":{"1min":0.94099999999999995,"5min":0.83899999999999997,"15min":0.77300000000000002},"max":{"1min":1.823,"5min":1.823,"15min":11.345000000000001},"last":1.0029999999999999}]},{"osd":5,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.0289999999999999,"15min":1.0549999999999999},"min":{"1min":0.76200000000000001,"5min":0.71699999999999997,"15min":0.70299999999999996},"max":{"1min":1.498,"5min":1.7470000000000001,"15min":2.0030000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2190000000000001,"15min":1.266},"min":{"1min":0.84399999999999997,"5min":0.82799999999999996,"15min":0.66300000000000003},"max":{"1min":1.4099999999999999,"5min":1.6539999999999999,"15min":2.1509999999999998},"last":0.93300000000000005}]},{"osd":6,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1819999999999999,"15min":1.2130000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.369,"5min":1.871,"15min":2.198},"last":1.04},{"interface":"front","average":{"1min":1.353,"5min":1.3540000000000001,"15min":1.397},"min":{"1min":1.0269999999999999,"5min":0.91000000000000003,"15min":0.872},"max":{"1min":1.74,"5min":1.8120000000000001,"15min":2.2509999999999999},"last":1.0820000000000001}]},{"osd":7,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.48,"5min":1.3680000000000001,"15min":1.4219999999999999},"min":{"1min":1.1839999999999999,"5min":0.96299999999999997,"15min":0.85799999999999998},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":11.374000000000001},"last":1.23},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3460000000000001,"15min":1.369},"min":{"1min":0.97899999999999998,"5min":0.94999999999999996,"15min":0.875},"max":{"1min":1.855,"5min":1.9810000000000001,"15min":2.3319999999999999},"last":1.173}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":27070464,"data_stored":27061270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22872064,"data_stored":22866980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12591104,"data_stored":12583584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":37752832,"data_stored":37749318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12599296,"data_stored":12584256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":25784320,"data_stored":25774643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":13201408,"data_stored":13191763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-21T22:52:43.236 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-01-21T22:52:43.236 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-21T22:52:43.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.514+0000 7fdc85400640 1 Processor -- start 2024-01-21T22:52:43.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.515+0000 7fdc85400640 1 -- start start 2024-01-21T22:52:43.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.516+0000 7fdc85400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fdc8015dab0 0x7fdc80165420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:43.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.516+0000 7fdc85400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc800c6630 con 0x7fdc80153ee0 2024-01-21T22:52:43.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.516+0000 7fdc85400640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc8006a850 con 0x7fdc800ca0d0 2024-01-21T22:52:43.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.516+0000 7fdc85400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc8015df00 con 0x7fdc8015dab0 2024-01-21T22:52:43.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.517+0000 7fdc7de00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fdc800ca0d0 0x7fdc800cade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:60470/0 (socket says 172.21.15.138:60470) 2024-01-21T22:52:43.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.517+0000 7fdc7de00640 1 -- 172.21.15.138:0/3357221303 learned_addr learned my addr 172.21.15.138:0/3357221303 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:52:43.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.517+0000 7fdc7f200640 1 --2- 172.21.15.138:0/3357221303 >> v2:172.21.15.138:3300/0 conn(0x7fdc8015dab0 0x7fdc80165420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:43.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.519+0000 7fdc7d400640 1 -- 172.21.15.138:0/3357221303 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1936622252 0 0) 0x7fdc8006a850 con 0x7fdc800ca0d0 2024-01-21T22:52:43.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.519+0000 7fdc7d400640 1 -- 172.21.15.138:0/3357221303 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc60003520 con 0x7fdc800ca0d0 2024-01-21T22:52:43.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.523+0000 7fdc7f200640 1 -- 172.21.15.138:0/3357221303 >> v1:172.21.15.152:6789/0 conn(0x7fdc800ca0d0 legacy=0x7fdc800cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:43.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.523+0000 7fdc7f200640 1 -- 172.21.15.138:0/3357221303 >> v1:172.21.15.138:6789/0 conn(0x7fdc80153ee0 legacy=0x7fdc8006bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:43.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.523+0000 7fdc7f200640 1 -- 172.21.15.138:0/3357221303 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc801542c0 con 0x7fdc8015dab0 2024-01-21T22:52:43.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.526+0000 7fdc7f200640 1 --2- 172.21.15.138:0/3357221303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc8015dab0 0x7fdc80165420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdc700097a0 tx=0x7fdc7002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7732e1147de44ff0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:43.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.532+0000 7fdc7d400640 1 -- 172.21.15.138:0/3357221303 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc70030610 con 0x7fdc8015dab0 2024-01-21T22:52:43.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.532+0000 7fdc7d400640 1 -- 172.21.15.138:0/3357221303 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc70030770 con 0x7fdc8015dab0 2024-01-21T22:52:43.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.532+0000 7fdc7d400640 1 -- 172.21.15.138:0/3357221303 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc70030b00 con 0x7fdc8015dab0 2024-01-21T22:52:43.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.533+0000 7fdc85400640 1 -- 172.21.15.138:0/3357221303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc8015dab0 msgr2=0x7fdc80165420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:43.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.533+0000 7fdc85400640 1 --2- 172.21.15.138:0/3357221303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc8015dab0 0x7fdc80165420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdc700097a0 tx=0x7fdc7002b960 comp rx=0 tx=0).stop 2024-01-21T22:52:43.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.533+0000 7fdc85400640 1 -- 172.21.15.138:0/3357221303 shutdown_connections 2024-01-21T22:52:43.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.533+0000 7fdc85400640 1 --2- 172.21.15.138:0/3357221303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc8015dab0 0x7fdc80165420 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:43.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.533+0000 7fdc85400640 1 -- 172.21.15.138:0/3357221303 >> 172.21.15.138:0/3357221303 conn(0x7fdc800cf410 msgr2=0x7fdc800cf820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:43.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.533+0000 7fdc85400640 1 -- 172.21.15.138:0/3357221303 shutdown_connections 2024-01-21T22:52:43.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.533+0000 7fdc85400640 1 -- 172.21.15.138:0/3357221303 wait complete. 2024-01-21T22:52:43.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.535+0000 7fdc85400640 1 Processor -- start 2024-01-21T22:52:43.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.535+0000 7fdc85400640 1 -- start start 2024-01-21T22:52:43.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.536+0000 7fdc85400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc800ca0d0 0x7fdc8015c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:43.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.536+0000 7fdc85400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc80153ee0 0x7fdc80159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:43.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.536+0000 7fdc85400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdc8015dab0 0x7fdc8015af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:43.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.536+0000 7fdc85400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc80158c00 con 0x7fdc80153ee0 2024-01-21T22:52:43.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.536+0000 7fdc85400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc8015cb90 con 0x7fdc800ca0d0 2024-01-21T22:52:43.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.536+0000 7fdc85400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc80157d70 con 0x7fdc8015dab0 2024-01-21T22:52:43.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.536+0000 7fdc7de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc80153ee0 0x7fdc80159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:43.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.537+0000 7fdc7e800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc800ca0d0 0x7fdc8015c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:43.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.537+0000 7fdc7f200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdc8015dab0 0x7fdc8015af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:43.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.537+0000 7fdc7de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc80153ee0 0x7fdc80159a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50908/0 (socket says 172.21.15.138:50908) 2024-01-21T22:52:43.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.537+0000 7fdc7de00640 1 -- 172.21.15.138:0/142868452 learned_addr learned my addr 172.21.15.138:0/142868452 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:43.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.541+0000 7fdc7de00640 1 -- 172.21.15.138:0/142868452 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdc8015dab0 msgr2=0x7fdc8015af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:43.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.541+0000 7fdc7de00640 1 --2- 172.21.15.138:0/142868452 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdc8015dab0 0x7fdc8015af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:43.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.541+0000 7fdc7de00640 1 -- 172.21.15.138:0/142868452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc800ca0d0 msgr2=0x7fdc8015c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:43.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.541+0000 7fdc7de00640 1 --2- 172.21.15.138:0/142868452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc800ca0d0 0x7fdc8015c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:43.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.541+0000 7fdc7de00640 1 -- 172.21.15.138:0/142868452 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc70009400 con 0x7fdc80153ee0 2024-01-21T22:52:43.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.543+0000 7fdc7de00640 1 --2- 172.21.15.138:0/142868452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc80153ee0 0x7fdc80159a70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdc6800cbf0 tx=0x7fdc6800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:43.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.547+0000 7fdc76a00640 1 -- 172.21.15.138:0/142868452 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6800ad50 con 0x7fdc80153ee0 2024-01-21T22:52:43.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.547+0000 7fdc76a00640 1 -- 172.21.15.138:0/142868452 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc68012400 con 0x7fdc80153ee0 2024-01-21T22:52:43.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.547+0000 7fdc76a00640 1 -- 172.21.15.138:0/142868452 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc68012790 con 0x7fdc80153ee0 2024-01-21T22:52:43.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.547+0000 7fdc85400640 1 -- 172.21.15.138:0/142868452 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc8015cea0 con 0x7fdc80153ee0 2024-01-21T22:52:43.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.548+0000 7fdc85400640 1 -- 172.21.15.138:0/142868452 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc8015d140 con 0x7fdc80153ee0 2024-01-21T22:52:43.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.549+0000 7fdc85400640 1 -- 172.21.15.138:0/142868452 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc800cfd00 con 0x7fdc80153ee0 2024-01-21T22:52:43.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.861+0000 7fdc76a00640 1 -- 172.21.15.138:0/142868452 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fdc68007070 con 0x7fdc80153ee0 2024-01-21T22:52:43.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.862+0000 7fdc76a00640 1 --2- 172.21.15.138:0/142868452 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fdc5806b3b0 0x7fdc5806d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:43.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.863+0000 7fdc7e800640 1 --2- 172.21.15.138:0/142868452 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fdc5806b3b0 0x7fdc5806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:43.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.864+0000 7fdc7e800640 1 --2- 172.21.15.138:0/142868452 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fdc5806b3b0 0x7fdc5806d870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdc78001780 tx=0x7fdc78009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:43.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.901+0000 7fdc76a00640 1 -- 172.21.15.138:0/142868452 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fdc68058ad0 con 0x7fdc80153ee0 2024-01-21T22:52:43.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:43.902+0000 7fdc76a00640 1 -- 172.21.15.138:0/142868452 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fdc68054e10 con 0x7fdc80153ee0 2024-01-21T22:52:44.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.105+0000 7fdc85400640 1 -- 172.21.15.138:0/142868452 --> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdc8006b060 con 0x7fdc5806b3b0 2024-01-21T22:52:44.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.113+0000 7fdc76a00640 1 -- 172.21.15.138:0/142868452 <== mgr.4111 v2:172.21.15.152:6800/314625564 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289901 (secure 0 0 0) 0x7fdc78001420 con 0x7fdc5806b3b0 2024-01-21T22:52:44.119 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:52:44.120 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-01-21T22:52:44.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.118+0000 7fdc85400640 1 -- 172.21.15.138:0/142868452 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fdc5806b3b0 msgr2=0x7fdc5806d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:44.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.118+0000 7fdc85400640 1 --2- 172.21.15.138:0/142868452 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fdc5806b3b0 0x7fdc5806d870 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdc78001780 tx=0x7fdc78009040 comp rx=0 tx=0).stop 2024-01-21T22:52:44.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.118+0000 7fdc85400640 1 -- 172.21.15.138:0/142868452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc80153ee0 msgr2=0x7fdc80159a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:44.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.118+0000 7fdc85400640 1 --2- 172.21.15.138:0/142868452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc80153ee0 0x7fdc80159a70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdc6800cbf0 tx=0x7fdc6800a430 comp rx=0 tx=0).stop 2024-01-21T22:52:44.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.118+0000 7fdc85400640 1 -- 172.21.15.138:0/142868452 shutdown_connections 2024-01-21T22:52:44.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.118+0000 7fdc85400640 1 --2- 172.21.15.138:0/142868452 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fdc5806b3b0 0x7fdc5806d870 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:44.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.118+0000 7fdc85400640 1 --2- 172.21.15.138:0/142868452 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fdc8015dab0 0x7fdc8015af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:44.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.118+0000 7fdc85400640 1 --2- 172.21.15.138:0/142868452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc80153ee0 0x7fdc80159a70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:44.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.118+0000 7fdc85400640 1 --2- 172.21.15.138:0/142868452 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fdc800ca0d0 0x7fdc8015c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:44.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.118+0000 7fdc85400640 1 -- 172.21.15.138:0/142868452 >> 172.21.15.138:0/142868452 conn(0x7fdc800cf410 msgr2=0x7fdc8014f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:44.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.119+0000 7fdc85400640 1 -- 172.21.15.138:0/142868452 shutdown_connections 2024-01-21T22:52:44.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.119+0000 7fdc85400640 1 -- 172.21.15.138:0/142868452 wait complete. 2024-01-21T22:52:44.141 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":859,"stamp":"2024-01-21T22:52:43.538074+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":82810866,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":1695,"num_read_kb":212798,"num_write":26823,"num_write_kb":599405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28153,"ondisk_log_size":28153,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":610384,"kb_used_data":163856,"kb_used_omap":12,"kb_used_meta":446451,"kb_avail":754364336,"statfs":{"total":773094113280,"available":772469080064,"internally_reserved":0,"allocated":167788544,"data_stored":166397260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":457166385},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":295667,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":13,"num_write":103,"num_write_kb":353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002501"},"pg_stats":[{"pgid":"3.3f","version":"25'165","reported_seq":200,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.428506+0000","last_change":"2024-01-21T22:25:04.888366+0000","last_active":"2024-01-21T22:52:41.428506+0000","last_peered":"2024-01-21T22:52:41.428506+0000","last_clean":"2024-01-21T22:52:41.428506+0000","last_became_active":"2024-01-21T22:25:04.888060+0000","last_became_peered":"2024-01-21T22:25:04.888060+0000","last_unstale":"2024-01-21T22:52:41.428506+0000","last_undegraded":"2024-01-21T22:52:41.428506+0000","last_fullsized":"2024-01-21T22:52:41.428506+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:32:13.512213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":115,"num_write_kb":6171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"25'170","reported_seq":210,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.411549+0000","last_change":"2024-01-21T22:25:04.843704+0000","last_active":"2024-01-21T22:52:41.411549+0000","last_peered":"2024-01-21T22:52:41.411549+0000","last_clean":"2024-01-21T22:52:41.411549+0000","last_became_active":"2024-01-21T22:25:04.843471+0000","last_became_peered":"2024-01-21T22:25:04.843471+0000","last_unstale":"2024-01-21T22:52:41.411549+0000","last_undegraded":"2024-01-21T22:52:41.411549+0000","last_fullsized":"2024-01-21T22:52:41.411549+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:34:24.822768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":129,"num_write_kb":4796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"25'202","reported_seq":241,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.418470+0000","last_change":"2024-01-21T22:25:04.858021+0000","last_active":"2024-01-21T22:52:41.418470+0000","last_peered":"2024-01-21T22:52:41.418470+0000","last_clean":"2024-01-21T22:52:41.418470+0000","last_became_active":"2024-01-21T22:25:04.857685+0000","last_became_peered":"2024-01-21T22:25:04.857685+0000","last_unstale":"2024-01-21T22:52:41.418470+0000","last_undegraded":"2024-01-21T22:52:41.418470+0000","last_fullsized":"2024-01-21T22:52:41.418470+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:29:02.950903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":157,"num_write_kb":6223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"25'180","reported_seq":215,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.361915+0000","last_change":"2024-01-21T22:25:04.858941+0000","last_active":"2024-01-21T22:52:41.361915+0000","last_peered":"2024-01-21T22:52:41.361915+0000","last_clean":"2024-01-21T22:52:41.361915+0000","last_became_active":"2024-01-21T22:25:04.858278+0000","last_became_peered":"2024-01-21T22:25:04.858278+0000","last_unstale":"2024-01-21T22:52:41.361915+0000","last_undegraded":"2024-01-21T22:52:41.361915+0000","last_fullsized":"2024-01-21T22:52:41.361915+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T10:03:50.093635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":135,"num_write_kb":19244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"25'181","reported_seq":218,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.422925+0000","last_change":"2024-01-21T22:25:04.881014+0000","last_active":"2024-01-21T22:52:41.422925+0000","last_peered":"2024-01-21T22:52:41.422925+0000","last_clean":"2024-01-21T22:52:41.422925+0000","last_became_active":"2024-01-21T22:25:04.878457+0000","last_became_peered":"2024-01-21T22:25:04.878457+0000","last_unstale":"2024-01-21T22:52:41.422925+0000","last_undegraded":"2024-01-21T22:52:41.422925+0000","last_fullsized":"2024-01-21T22:52:41.422925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:34:24.822768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":119,"num_write_kb":7041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"25'199","reported_seq":227,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.357140+0000","last_change":"2024-01-21T22:25:04.859776+0000","last_active":"2024-01-21T22:52:41.357140+0000","last_peered":"2024-01-21T22:52:41.357140+0000","last_clean":"2024-01-21T22:52:41.357140+0000","last_became_active":"2024-01-21T22:25:04.858731+0000","last_became_peered":"2024-01-21T22:25:04.858731+0000","last_unstale":"2024-01-21T22:52:41.357140+0000","last_undegraded":"2024-01-21T22:52:41.357140+0000","last_fullsized":"2024-01-21T22:52:41.357140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:01:30.091588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":148,"num_write_kb":2956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"25'212","reported_seq":256,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.425194+0000","last_change":"2024-01-21T22:25:04.881392+0000","last_active":"2024-01-21T22:52:41.425194+0000","last_peered":"2024-01-21T22:52:41.425194+0000","last_clean":"2024-01-21T22:52:41.425194+0000","last_became_active":"2024-01-21T22:25:04.881090+0000","last_became_peered":"2024-01-21T22:25:04.881090+0000","last_unstale":"2024-01-21T22:52:41.425194+0000","last_undegraded":"2024-01-21T22:52:41.425194+0000","last_fullsized":"2024-01-21T22:52:41.425194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:25:50.380424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":4572,"num_write":149,"num_write_kb":5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"25'174","reported_seq":213,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.421548+0000","last_change":"2024-01-21T22:25:04.879186+0000","last_active":"2024-01-21T22:52:41.421548+0000","last_peered":"2024-01-21T22:52:41.421548+0000","last_clean":"2024-01-21T22:52:41.421548+0000","last_became_active":"2024-01-21T22:25:04.878824+0000","last_became_peered":"2024-01-21T22:25:04.878824+0000","last_unstale":"2024-01-21T22:52:41.421548+0000","last_undegraded":"2024-01-21T22:52:41.421548+0000","last_fullsized":"2024-01-21T22:52:41.421548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:31:25.985811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":137,"num_write_kb":11909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"25'162","reported_seq":195,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.445529+0000","last_change":"2024-01-21T22:25:04.886391+0000","last_active":"2024-01-21T22:52:41.445529+0000","last_peered":"2024-01-21T22:52:41.445529+0000","last_clean":"2024-01-21T22:52:41.445529+0000","last_became_active":"2024-01-21T22:25:04.884979+0000","last_became_peered":"2024-01-21T22:25:04.884979+0000","last_unstale":"2024-01-21T22:52:41.445529+0000","last_undegraded":"2024-01-21T22:52:41.445529+0000","last_fullsized":"2024-01-21T22:52:41.445529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:36:36.009632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":96,"num_write_kb":3924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"25'180","reported_seq":219,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.430115+0000","last_change":"2024-01-21T22:25:04.888126+0000","last_active":"2024-01-21T22:52:41.430115+0000","last_peered":"2024-01-21T22:52:41.430115+0000","last_clean":"2024-01-21T22:52:41.430115+0000","last_became_active":"2024-01-21T22:25:04.887631+0000","last_became_peered":"2024-01-21T22:25:04.887631+0000","last_unstale":"2024-01-21T22:52:41.430115+0000","last_undegraded":"2024-01-21T22:52:41.430115+0000","last_fullsized":"2024-01-21T22:52:41.430115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:00:02.684745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":122,"num_write_kb":9871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"25'192","reported_seq":226,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.429276+0000","last_change":"2024-01-21T22:25:04.885677+0000","last_active":"2024-01-21T22:52:41.429276+0000","last_peered":"2024-01-21T22:52:41.429276+0000","last_clean":"2024-01-21T22:52:41.429276+0000","last_became_active":"2024-01-21T22:25:04.884697+0000","last_became_peered":"2024-01-21T22:25:04.884697+0000","last_unstale":"2024-01-21T22:52:41.429276+0000","last_undegraded":"2024-01-21T22:52:41.429276+0000","last_fullsized":"2024-01-21T22:52:41.429276+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:44:23.749631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":143,"num_write_kb":7172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"25'146","reported_seq":183,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.433880+0000","last_change":"2024-01-21T22:25:04.886799+0000","last_active":"2024-01-21T22:52:41.433880+0000","last_peered":"2024-01-21T22:52:41.433880+0000","last_clean":"2024-01-21T22:52:41.433880+0000","last_became_active":"2024-01-21T22:25:04.886387+0000","last_became_peered":"2024-01-21T22:25:04.886387+0000","last_unstale":"2024-01-21T22:52:41.433880+0000","last_undegraded":"2024-01-21T22:52:41.433880+0000","last_fullsized":"2024-01-21T22:52:41.433880+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:42:39.977125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":92,"num_write_kb":3768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"25'169","reported_seq":202,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.350337+0000","last_change":"2024-01-21T22:25:04.888103+0000","last_active":"2024-01-21T22:52:41.350337+0000","last_peered":"2024-01-21T22:52:41.350337+0000","last_clean":"2024-01-21T22:52:41.350337+0000","last_became_active":"2024-01-21T22:25:04.887908+0000","last_became_peered":"2024-01-21T22:25:04.887908+0000","last_unstale":"2024-01-21T22:52:41.350337+0000","last_undegraded":"2024-01-21T22:52:41.350337+0000","last_fullsized":"2024-01-21T22:52:41.350337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:23:52.444277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":122,"num_write_kb":8957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"25'174","reported_seq":213,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.343215+0000","last_change":"2024-01-21T22:25:04.842750+0000","last_active":"2024-01-21T22:52:41.343215+0000","last_peered":"2024-01-21T22:52:41.343215+0000","last_clean":"2024-01-21T22:52:41.343215+0000","last_became_active":"2024-01-21T22:25:04.842512+0000","last_became_peered":"2024-01-21T22:25:04.842512+0000","last_unstale":"2024-01-21T22:52:41.343215+0000","last_undegraded":"2024-01-21T22:52:41.343215+0000","last_fullsized":"2024-01-21T22:52:41.343215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:40:05.554704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":120,"num_write_kb":6061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"25'170","reported_seq":226,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.420873+0000","last_change":"2024-01-21T22:25:04.876592+0000","last_active":"2024-01-21T22:52:41.420873+0000","last_peered":"2024-01-21T22:52:41.420873+0000","last_clean":"2024-01-21T22:52:41.420873+0000","last_became_active":"2024-01-21T22:25:04.876305+0000","last_became_peered":"2024-01-21T22:25:04.876305+0000","last_unstale":"2024-01-21T22:52:41.420873+0000","last_undegraded":"2024-01-21T22:52:41.420873+0000","last_fullsized":"2024-01-21T22:52:41.420873+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:32.177562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":7620,"num_write":119,"num_write_kb":10742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"25'209","reported_seq":255,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.403552+0000","last_change":"2024-01-21T22:25:04.878038+0000","last_active":"2024-01-21T22:52:41.403552+0000","last_peered":"2024-01-21T22:52:41.403552+0000","last_clean":"2024-01-21T22:52:41.403552+0000","last_became_active":"2024-01-21T22:25:04.877716+0000","last_became_peered":"2024-01-21T22:25:04.877716+0000","last_unstale":"2024-01-21T22:52:41.403552+0000","last_undegraded":"2024-01-21T22:52:41.403552+0000","last_fullsized":"2024-01-21T22:52:41.403552+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:32:13.512213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":5080,"num_write":159,"num_write_kb":7672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"25'183","reported_seq":220,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.366489+0000","last_change":"2024-01-21T22:25:04.890537+0000","last_active":"2024-01-21T22:52:41.366489+0000","last_peered":"2024-01-21T22:52:41.366489+0000","last_clean":"2024-01-21T22:52:41.366489+0000","last_became_active":"2024-01-21T22:25:04.890298+0000","last_became_peered":"2024-01-21T22:25:04.890298+0000","last_unstale":"2024-01-21T22:52:41.366489+0000","last_undegraded":"2024-01-21T22:52:41.366489+0000","last_fullsized":"2024-01-21T22:52:41.366489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:23:57.435028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":136,"num_write_kb":7248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"25'159","reported_seq":194,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.349490+0000","last_change":"2024-01-21T22:25:04.712868+0000","last_active":"2024-01-21T22:52:41.349490+0000","last_peered":"2024-01-21T22:52:41.349490+0000","last_clean":"2024-01-21T22:52:41.349490+0000","last_became_active":"2024-01-21T22:25:04.712616+0000","last_became_peered":"2024-01-21T22:25:04.712616+0000","last_unstale":"2024-01-21T22:52:41.349490+0000","last_undegraded":"2024-01-21T22:52:41.349490+0000","last_fullsized":"2024-01-21T22:52:41.349490+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:50:41.306582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":102,"num_write_kb":4431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"25'163","reported_seq":203,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.359652+0000","last_change":"2024-01-21T22:25:04.879328+0000","last_active":"2024-01-21T22:52:41.359652+0000","last_peered":"2024-01-21T22:52:41.359652+0000","last_clean":"2024-01-21T22:52:41.359652+0000","last_became_active":"2024-01-21T22:25:04.879011+0000","last_became_peered":"2024-01-21T22:25:04.879011+0000","last_unstale":"2024-01-21T22:52:41.359652+0000","last_undegraded":"2024-01-21T22:52:41.359652+0000","last_fullsized":"2024-01-21T22:52:41.359652+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:31:23.569672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":108,"num_write_kb":6073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"25'243","reported_seq":284,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.434131+0000","last_change":"2024-01-21T22:25:04.885620+0000","last_active":"2024-01-21T22:52:41.434131+0000","last_peered":"2024-01-21T22:52:41.434131+0000","last_clean":"2024-01-21T22:52:41.434131+0000","last_became_active":"2024-01-21T22:25:04.884364+0000","last_became_peered":"2024-01-21T22:25:04.884364+0000","last_unstale":"2024-01-21T22:52:41.434131+0000","last_undegraded":"2024-01-21T22:52:41.434131+0000","last_fullsized":"2024-01-21T22:52:41.434131+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:13:04.601911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3556,"num_write":191,"num_write_kb":16584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"25'200","reported_seq":239,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.445400+0000","last_change":"2024-01-21T22:25:04.868371+0000","last_active":"2024-01-21T22:52:41.445400+0000","last_peered":"2024-01-21T22:52:41.445400+0000","last_clean":"2024-01-21T22:52:41.445400+0000","last_became_active":"2024-01-21T22:25:04.868196+0000","last_became_peered":"2024-01-21T22:25:04.868196+0000","last_unstale":"2024-01-21T22:52:41.445400+0000","last_undegraded":"2024-01-21T22:52:41.445400+0000","last_fullsized":"2024-01-21T22:52:41.445400+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:05:41.950232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":139,"num_write_kb":3732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"25'203","reported_seq":260,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.446332+0000","last_change":"2024-01-21T22:25:04.879144+0000","last_active":"2024-01-21T22:52:41.446332+0000","last_peered":"2024-01-21T22:52:41.446332+0000","last_clean":"2024-01-21T22:52:41.446332+0000","last_became_active":"2024-01-21T22:25:04.878868+0000","last_became_peered":"2024-01-21T22:25:04.878868+0000","last_unstale":"2024-01-21T22:52:41.446332+0000","last_undegraded":"2024-01-21T22:52:41.446332+0000","last_fullsized":"2024-01-21T22:52:41.446332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:55:28.069411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":7620,"num_write":153,"num_write_kb":6732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"25'173","reported_seq":209,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.419925+0000","last_change":"2024-01-21T22:25:04.715246+0000","last_active":"2024-01-21T22:52:41.419925+0000","last_peered":"2024-01-21T22:52:41.419925+0000","last_clean":"2024-01-21T22:52:41.419925+0000","last_became_active":"2024-01-21T22:25:04.714969+0000","last_became_peered":"2024-01-21T22:25:04.714969+0000","last_unstale":"2024-01-21T22:52:41.419925+0000","last_undegraded":"2024-01-21T22:52:41.419925+0000","last_fullsized":"2024-01-21T22:52:41.419925+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:56:24.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":106,"num_write_kb":4123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"25'182","reported_seq":230,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.401975+0000","last_change":"2024-01-21T22:25:04.877902+0000","last_active":"2024-01-21T22:52:41.401975+0000","last_peered":"2024-01-21T22:52:41.401975+0000","last_clean":"2024-01-21T22:52:41.401975+0000","last_became_active":"2024-01-21T22:25:04.876963+0000","last_became_peered":"2024-01-21T22:25:04.876963+0000","last_unstale":"2024-01-21T22:52:41.401975+0000","last_undegraded":"2024-01-21T22:52:41.401975+0000","last_fullsized":"2024-01-21T22:52:41.401975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:30:48.864454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":5334,"num_write":137,"num_write_kb":8161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"25'184","reported_seq":234,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.418734+0000","last_change":"2024-01-21T22:25:04.716735+0000","last_active":"2024-01-21T22:52:41.418734+0000","last_peered":"2024-01-21T22:52:41.418734+0000","last_clean":"2024-01-21T22:52:41.418734+0000","last_became_active":"2024-01-21T22:25:04.716570+0000","last_became_peered":"2024-01-21T22:25:04.716570+0000","last_unstale":"2024-01-21T22:52:41.418734+0000","last_undegraded":"2024-01-21T22:52:41.418734+0000","last_fullsized":"2024-01-21T22:52:41.418734+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:44:55.158990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":5842,"num_write":145,"num_write_kb":5757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"25'176","reported_seq":217,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.368001+0000","last_change":"2024-01-21T22:25:04.886230+0000","last_active":"2024-01-21T22:52:41.368001+0000","last_peered":"2024-01-21T22:52:41.368001+0000","last_clean":"2024-01-21T22:52:41.368001+0000","last_became_active":"2024-01-21T22:25:04.885523+0000","last_became_peered":"2024-01-21T22:25:04.885523+0000","last_unstale":"2024-01-21T22:52:41.368001+0000","last_undegraded":"2024-01-21T22:52:41.368001+0000","last_fullsized":"2024-01-21T22:52:41.368001+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:06:02.934920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3556,"num_write":106,"num_write_kb":5820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"25'213","reported_seq":249,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.346332+0000","last_change":"2024-01-21T22:25:04.842692+0000","last_active":"2024-01-21T22:52:41.346332+0000","last_peered":"2024-01-21T22:52:41.346332+0000","last_clean":"2024-01-21T22:52:41.346332+0000","last_became_active":"2024-01-21T22:25:04.842429+0000","last_became_peered":"2024-01-21T22:25:04.842429+0000","last_unstale":"2024-01-21T22:52:41.346332+0000","last_undegraded":"2024-01-21T22:52:41.346332+0000","last_fullsized":"2024-01-21T22:52:41.346332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:52:24.233709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":167,"num_write_kb":10319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"25'183","reported_seq":222,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.350241+0000","last_change":"2024-01-21T22:25:04.888296+0000","last_active":"2024-01-21T22:52:41.350241+0000","last_peered":"2024-01-21T22:52:41.350241+0000","last_clean":"2024-01-21T22:52:41.350241+0000","last_became_active":"2024-01-21T22:25:04.888025+0000","last_became_peered":"2024-01-21T22:25:04.888025+0000","last_unstale":"2024-01-21T22:52:41.350241+0000","last_undegraded":"2024-01-21T22:52:41.350241+0000","last_fullsized":"2024-01-21T22:52:41.350241+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:57:44.129490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":136,"num_write_kb":6753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"25'199","reported_seq":239,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.416957+0000","last_change":"2024-01-21T22:25:04.711908+0000","last_active":"2024-01-21T22:52:41.416957+0000","last_peered":"2024-01-21T22:52:41.416957+0000","last_clean":"2024-01-21T22:52:41.416957+0000","last_became_active":"2024-01-21T22:25:04.711654+0000","last_became_peered":"2024-01-21T22:25:04.711654+0000","last_unstale":"2024-01-21T22:52:41.416957+0000","last_undegraded":"2024-01-21T22:52:41.416957+0000","last_fullsized":"2024-01-21T22:52:41.416957+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:20:41.612265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":148,"num_write_kb":7328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"25'334","reported_seq":375,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.357261+0000","last_change":"2024-01-21T22:25:04.876757+0000","last_active":"2024-01-21T22:52:41.357261+0000","last_peered":"2024-01-21T22:52:41.357261+0000","last_clean":"2024-01-21T22:52:41.357261+0000","last_became_active":"2024-01-21T22:25:04.876012+0000","last_became_peered":"2024-01-21T22:25:04.876012+0000","last_unstale":"2024-01-21T22:52:41.357261+0000","last_undegraded":"2024-01-21T22:52:41.357261+0000","last_fullsized":"2024-01-21T22:52:41.357261+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:00:56.239773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":275,"num_write_kb":7140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"25'207","reported_seq":243,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.342299+0000","last_change":"2024-01-21T22:25:04.715277+0000","last_active":"2024-01-21T22:52:41.342299+0000","last_peered":"2024-01-21T22:52:41.342299+0000","last_clean":"2024-01-21T22:52:41.342299+0000","last_became_active":"2024-01-21T22:25:04.715034+0000","last_became_peered":"2024-01-21T22:25:04.715034+0000","last_unstale":"2024-01-21T22:52:41.342299+0000","last_undegraded":"2024-01-21T22:52:41.342299+0000","last_fullsized":"2024-01-21T22:52:41.342299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:55:28.069411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":140,"num_write_kb":6049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"25'210","reported_seq":242,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.357216+0000","last_change":"2024-01-21T22:25:04.844472+0000","last_active":"2024-01-21T22:52:41.357216+0000","last_peered":"2024-01-21T22:52:41.357216+0000","last_clean":"2024-01-21T22:52:41.357216+0000","last_became_active":"2024-01-21T22:25:04.843827+0000","last_became_peered":"2024-01-21T22:25:04.843827+0000","last_unstale":"2024-01-21T22:52:41.357216+0000","last_undegraded":"2024-01-21T22:52:41.357216+0000","last_fullsized":"2024-01-21T22:52:41.357216+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:58:10.552987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1016,"num_write":149,"num_write_kb":12027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"25'158","reported_seq":195,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.444472+0000","last_change":"2024-01-21T22:25:04.713764+0000","last_active":"2024-01-21T22:52:41.444472+0000","last_peered":"2024-01-21T22:52:41.444472+0000","last_clean":"2024-01-21T22:52:41.444472+0000","last_became_active":"2024-01-21T22:25:04.712814+0000","last_became_peered":"2024-01-21T22:25:04.712814+0000","last_unstale":"2024-01-21T22:52:41.444472+0000","last_undegraded":"2024-01-21T22:52:41.444472+0000","last_fullsized":"2024-01-21T22:52:41.444472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:16:41.440927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":111,"num_write_kb":4009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"25'753","reported_seq":780,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:39.513204+0000","last_change":"2024-01-21T22:25:03.703079+0000","last_active":"2024-01-21T22:52:39.513204+0000","last_peered":"2024-01-21T22:52:39.513204+0000","last_clean":"2024-01-21T22:52:39.513204+0000","last_became_active":"2024-01-21T22:25:03.702776+0000","last_became_peered":"2024-01-21T22:25:03.702776+0000","last_unstale":"2024-01-21T22:52:39.513204+0000","last_undegraded":"2024-01-21T22:52:39.513204+0000","last_fullsized":"2024-01-21T22:52:39.513204+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":753,"log_dups_size":0,"ondisk_log_size":753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:34:13.302149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":748,"num_write_kb":4569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"25'198","reported_seq":235,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.445514+0000","last_change":"2024-01-21T22:25:04.877158+0000","last_active":"2024-01-21T22:52:41.445514+0000","last_peered":"2024-01-21T22:52:41.445514+0000","last_clean":"2024-01-21T22:52:41.445514+0000","last_became_active":"2024-01-21T22:25:04.876920+0000","last_became_peered":"2024-01-21T22:25:04.876920+0000","last_unstale":"2024-01-21T22:52:41.445514+0000","last_undegraded":"2024-01-21T22:52:41.445514+0000","last_fullsized":"2024-01-21T22:52:41.445514+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:46:55.451173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":152,"num_write_kb":6158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"25'776","reported_seq":803,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.474218+0000","last_change":"2024-01-21T22:25:03.720178+0000","last_active":"2024-01-21T22:52:33.474218+0000","last_peered":"2024-01-21T22:52:33.474218+0000","last_clean":"2024-01-21T22:52:33.474218+0000","last_became_active":"2024-01-21T22:25:03.719909+0000","last_became_peered":"2024-01-21T22:25:03.719909+0000","last_unstale":"2024-01-21T22:52:33.474218+0000","last_undegraded":"2024-01-21T22:52:33.474218+0000","last_fullsized":"2024-01-21T22:52:33.474218+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:44:53.910872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":769,"num_write_kb":4580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"25'3","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.462701+0000","last_change":"2024-01-21T22:25:03.723005+0000","last_active":"2024-01-21T22:52:28.462701+0000","last_peered":"2024-01-21T22:52:28.462701+0000","last_clean":"2024-01-21T22:52:28.462701+0000","last_became_active":"2024-01-21T22:25:03.722758+0000","last_became_peered":"2024-01-21T22:25:03.722758+0000","last_unstale":"2024-01-21T22:52:28.462701+0000","last_undegraded":"2024-01-21T22:52:28.462701+0000","last_fullsized":"2024-01-21T22:52:28.462701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:16:40.192808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"25'224","reported_seq":263,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.418498+0000","last_change":"2024-01-21T22:25:04.844030+0000","last_active":"2024-01-21T22:52:41.418498+0000","last_peered":"2024-01-21T22:52:41.418498+0000","last_clean":"2024-01-21T22:52:41.418498+0000","last_became_active":"2024-01-21T22:25:04.843714+0000","last_became_peered":"2024-01-21T22:25:04.843714+0000","last_unstale":"2024-01-21T22:52:41.418498+0000","last_undegraded":"2024-01-21T22:52:41.418498+0000","last_fullsized":"2024-01-21T22:52:41.418498+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:25:50.380424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":174,"num_write_kb":5546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"25'1195","reported_seq":1222,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.505239+0000","last_change":"2024-01-21T22:25:03.702767+0000","last_active":"2024-01-21T22:52:23.505239+0000","last_peered":"2024-01-21T22:52:23.505239+0000","last_clean":"2024-01-21T22:52:23.505239+0000","last_became_active":"2024-01-21T22:25:03.702140+0000","last_became_peered":"2024-01-21T22:25:03.702140+0000","last_unstale":"2024-01-21T22:52:23.505239+0000","last_undegraded":"2024-01-21T22:52:23.505239+0000","last_fullsized":"2024-01-21T22:52:23.505239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:46:54.203055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1190,"num_write_kb":8787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"25'199","reported_seq":237,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.446004+0000","last_change":"2024-01-21T22:25:04.867173+0000","last_active":"2024-01-21T22:52:41.446004+0000","last_peered":"2024-01-21T22:52:41.446004+0000","last_clean":"2024-01-21T22:52:41.446004+0000","last_became_active":"2024-01-21T22:25:04.866945+0000","last_became_peered":"2024-01-21T22:25:04.866945+0000","last_unstale":"2024-01-21T22:52:41.446004+0000","last_undegraded":"2024-01-21T22:52:41.446004+0000","last_fullsized":"2024-01-21T22:52:41.446004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:15:41.725132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":136,"num_write_kb":8996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.462303+0000","last_change":"2024-01-21T22:25:03.715128+0000","last_active":"2024-01-21T22:52:23.462303+0000","last_peered":"2024-01-21T22:52:23.462303+0000","last_clean":"2024-01-21T22:52:23.462303+0000","last_became_active":"2024-01-21T22:25:03.714039+0000","last_became_peered":"2024-01-21T22:25:03.714039+0000","last_unstale":"2024-01-21T22:52:23.462303+0000","last_undegraded":"2024-01-21T22:52:23.462303+0000","last_fullsized":"2024-01-21T22:52:23.462303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:09:09.988399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"25'210","reported_seq":248,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.369210+0000","last_change":"2024-01-21T22:25:04.886534+0000","last_active":"2024-01-21T22:52:41.369210+0000","last_peered":"2024-01-21T22:52:41.369210+0000","last_clean":"2024-01-21T22:52:41.369210+0000","last_became_active":"2024-01-21T22:25:04.885963+0000","last_became_peered":"2024-01-21T22:25:04.885963+0000","last_unstale":"2024-01-21T22:52:41.369210+0000","last_undegraded":"2024-01-21T22:52:41.369210+0000","last_fullsized":"2024-01-21T22:52:41.369210+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:44:31.867220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":149,"num_write_kb":6741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"25'5","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.579983+0000","last_change":"2024-01-21T22:25:03.718237+0000","last_active":"2024-01-21T22:52:33.579983+0000","last_peered":"2024-01-21T22:52:33.579983+0000","last_clean":"2024-01-21T22:52:33.579983+0000","last_became_active":"2024-01-21T22:25:03.717937+0000","last_became_peered":"2024-01-21T22:25:03.717937+0000","last_unstale":"2024-01-21T22:52:33.579983+0000","last_undegraded":"2024-01-21T22:52:33.579983+0000","last_fullsized":"2024-01-21T22:52:33.579983+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:31:24.737692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"25'195","reported_seq":231,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.418390+0000","last_change":"2024-01-21T22:25:04.882743+0000","last_active":"2024-01-21T22:52:41.418390+0000","last_peered":"2024-01-21T22:52:41.418390+0000","last_clean":"2024-01-21T22:52:41.418390+0000","last_became_active":"2024-01-21T22:25:04.882478+0000","last_became_peered":"2024-01-21T22:25:04.882478+0000","last_unstale":"2024-01-21T22:52:41.418390+0000","last_undegraded":"2024-01-21T22:52:41.418390+0000","last_fullsized":"2024-01-21T22:52:41.418390+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:42:39.977125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2540,"num_write":148,"num_write_kb":5415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"25'468","reported_seq":495,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.588185+0000","last_change":"2024-01-21T22:25:03.703672+0000","last_active":"2024-01-21T22:52:33.588185+0000","last_peered":"2024-01-21T22:52:33.588185+0000","last_clean":"2024-01-21T22:52:33.588185+0000","last_became_active":"2024-01-21T22:25:03.703426+0000","last_became_peered":"2024-01-21T22:25:03.703426+0000","last_unstale":"2024-01-21T22:52:33.588185+0000","last_undegraded":"2024-01-21T22:52:33.588185+0000","last_fullsized":"2024-01-21T22:52:33.588185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:41:51.793206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":467,"num_write_kb":464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"25'215","reported_seq":263,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.415421+0000","last_change":"2024-01-21T22:25:04.887306+0000","last_active":"2024-01-21T22:52:41.415421+0000","last_peered":"2024-01-21T22:52:41.415421+0000","last_clean":"2024-01-21T22:52:41.415421+0000","last_became_active":"2024-01-21T22:25:04.886970+0000","last_became_peered":"2024-01-21T22:25:04.886970+0000","last_unstale":"2024-01-21T22:52:41.415421+0000","last_undegraded":"2024-01-21T22:52:41.415421+0000","last_fullsized":"2024-01-21T22:52:41.415421+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:59:45.239426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":5588,"num_write":173,"num_write_kb":7144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"25'879","reported_seq":906,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:34.208691+0000","last_change":"2024-01-21T22:25:03.714102+0000","last_active":"2024-01-21T22:52:34.208691+0000","last_peered":"2024-01-21T22:52:34.208691+0000","last_clean":"2024-01-21T22:52:34.208691+0000","last_became_active":"2024-01-21T22:25:03.713717+0000","last_became_peered":"2024-01-21T22:25:03.713717+0000","last_unstale":"2024-01-21T22:52:34.208691+0000","last_undegraded":"2024-01-21T22:52:34.208691+0000","last_fullsized":"2024-01-21T22:52:34.208691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":879,"log_dups_size":0,"ondisk_log_size":879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:32:15.181238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":877,"num_write_kb":4705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"25'169","reported_seq":213,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.426022+0000","last_change":"2024-01-21T22:25:04.881307+0000","last_active":"2024-01-21T22:52:41.426022+0000","last_peered":"2024-01-21T22:52:41.426022+0000","last_clean":"2024-01-21T22:52:41.426022+0000","last_became_active":"2024-01-21T22:25:04.880472+0000","last_became_peered":"2024-01-21T22:25:04.880472+0000","last_unstale":"2024-01-21T22:52:41.426022+0000","last_undegraded":"2024-01-21T22:52:41.426022+0000","last_fullsized":"2024-01-21T22:52:41.426022+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:29:02.950903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":4572,"num_write":122,"num_write_kb":6796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"25'1290","reported_seq":1317,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.495049+0000","last_change":"2024-01-21T22:25:03.721825+0000","last_active":"2024-01-21T22:52:33.495049+0000","last_peered":"2024-01-21T22:52:33.495049+0000","last_clean":"2024-01-21T22:52:33.495049+0000","last_became_active":"2024-01-21T22:25:03.721502+0000","last_became_peered":"2024-01-21T22:25:03.721502+0000","last_unstale":"2024-01-21T22:52:33.495049+0000","last_undegraded":"2024-01-21T22:52:33.495049+0000","last_fullsized":"2024-01-21T22:52:33.495049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1290,"log_dups_size":0,"ondisk_log_size":1290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:09:09.988399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1286,"num_write_kb":8849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"25'164","reported_seq":195,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.350536+0000","last_change":"2024-01-21T22:25:04.877455+0000","last_active":"2024-01-21T22:52:41.350536+0000","last_peered":"2024-01-21T22:52:41.350536+0000","last_clean":"2024-01-21T22:52:41.350536+0000","last_became_active":"2024-01-21T22:25:04.877257+0000","last_became_peered":"2024-01-21T22:25:04.877257+0000","last_unstale":"2024-01-21T22:52:41.350536+0000","last_undegraded":"2024-01-21T22:52:41.350536+0000","last_fullsized":"2024-01-21T22:52:41.350536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:34:14.550267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1270,"num_write":117,"num_write_kb":4465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"25'5","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.529940+0000","last_change":"2024-01-21T22:25:03.706077+0000","last_active":"2024-01-21T22:52:33.529940+0000","last_peered":"2024-01-21T22:52:33.529940+0000","last_clean":"2024-01-21T22:52:33.529940+0000","last_became_active":"2024-01-21T22:25:03.702798+0000","last_became_peered":"2024-01-21T22:25:03.702798+0000","last_unstale":"2024-01-21T22:52:33.529940+0000","last_undegraded":"2024-01-21T22:52:33.529940+0000","last_fullsized":"2024-01-21T22:52:33.529940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:20:40.364147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"25'231","reported_seq":266,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.410049+0000","last_change":"2024-01-21T22:25:04.890636+0000","last_active":"2024-01-21T22:52:41.410049+0000","last_peered":"2024-01-21T22:52:41.410049+0000","last_clean":"2024-01-21T22:52:41.410049+0000","last_became_active":"2024-01-21T22:25:04.890271+0000","last_became_peered":"2024-01-21T22:25:04.890271+0000","last_unstale":"2024-01-21T22:52:41.410049+0000","last_undegraded":"2024-01-21T22:52:41.410049+0000","last_fullsized":"2024-01-21T22:52:41.410049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:15:46.754395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":174,"num_write_kb":31551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"25'4","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.470849+0000","last_change":"2024-01-21T22:25:03.717202+0000","last_active":"2024-01-21T22:52:28.470849+0000","last_peered":"2024-01-21T22:52:28.470849+0000","last_clean":"2024-01-21T22:52:28.470849+0000","last_became_active":"2024-01-21T22:25:03.716846+0000","last_became_peered":"2024-01-21T22:25:03.716846+0000","last_unstale":"2024-01-21T22:52:28.470849+0000","last_undegraded":"2024-01-21T22:52:28.470849+0000","last_fullsized":"2024-01-21T22:52:28.470849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:10.904923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"25'169","reported_seq":202,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.436305+0000","last_change":"2024-01-21T22:25:04.715053+0000","last_active":"2024-01-21T22:52:41.436305+0000","last_peered":"2024-01-21T22:52:41.436305+0000","last_clean":"2024-01-21T22:52:41.436305+0000","last_became_active":"2024-01-21T22:25:04.713566+0000","last_became_peered":"2024-01-21T22:25:04.713566+0000","last_unstale":"2024-01-21T22:52:41.436305+0000","last_undegraded":"2024-01-21T22:52:41.436305+0000","last_fullsized":"2024-01-21T22:52:41.436305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:50:41.306582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":119,"num_write_kb":3558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"25'10","reported_seq":37,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.538682+0000","last_change":"2024-01-21T22:25:03.703017+0000","last_active":"2024-01-21T22:52:33.538682+0000","last_peered":"2024-01-21T22:52:33.538682+0000","last_clean":"2024-01-21T22:52:33.538682+0000","last_became_active":"2024-01-21T22:25:03.702742+0000","last_became_peered":"2024-01-21T22:25:03.702742+0000","last_unstale":"2024-01-21T22:52:33.538682+0000","last_undegraded":"2024-01-21T22:52:33.538682+0000","last_fullsized":"2024-01-21T22:52:33.538682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:42:03.386594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"25'196","reported_seq":229,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.363935+0000","last_change":"2024-01-21T22:25:04.876507+0000","last_active":"2024-01-21T22:52:41.363935+0000","last_peered":"2024-01-21T22:52:41.363935+0000","last_clean":"2024-01-21T22:52:41.363935+0000","last_became_active":"2024-01-21T22:25:04.876169+0000","last_became_peered":"2024-01-21T22:25:04.876169+0000","last_unstale":"2024-01-21T22:52:41.363935+0000","last_undegraded":"2024-01-21T22:52:41.363935+0000","last_fullsized":"2024-01-21T22:52:41.363935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:12:11.958115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":142,"num_write_kb":8431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"25'7","reported_seq":35,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.467670+0000","last_change":"2024-01-21T22:25:03.720257+0000","last_active":"2024-01-21T22:52:28.467670+0000","last_peered":"2024-01-21T22:52:28.467670+0000","last_clean":"2024-01-21T22:52:28.467670+0000","last_became_active":"2024-01-21T22:25:03.720009+0000","last_became_peered":"2024-01-21T22:25:03.720009+0000","last_unstale":"2024-01-21T22:52:28.467670+0000","last_undegraded":"2024-01-21T22:52:28.467670+0000","last_fullsized":"2024-01-21T22:52:28.467670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:37:56.569677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"25'153","reported_seq":201,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.368279+0000","last_change":"2024-01-21T22:25:04.883112+0000","last_active":"2024-01-21T22:52:41.368279+0000","last_peered":"2024-01-21T22:52:41.368279+0000","last_clean":"2024-01-21T22:52:41.368279+0000","last_became_active":"2024-01-21T22:25:04.882767+0000","last_became_peered":"2024-01-21T22:25:04.882767+0000","last_unstale":"2024-01-21T22:52:41.368279+0000","last_undegraded":"2024-01-21T22:52:41.368279+0000","last_fullsized":"2024-01-21T22:52:41.368279+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:18:03.439738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":5334,"num_write":115,"num_write_kb":10137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"25'300","reported_seq":328,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.539211+0000","last_change":"2024-01-21T22:25:03.703516+0000","last_active":"2024-01-21T22:52:33.539211+0000","last_peered":"2024-01-21T22:52:33.539211+0000","last_clean":"2024-01-21T22:52:33.539211+0000","last_became_active":"2024-01-21T22:25:03.703256+0000","last_became_peered":"2024-01-21T22:25:03.703256+0000","last_unstale":"2024-01-21T22:52:33.539211+0000","last_undegraded":"2024-01-21T22:52:33.539211+0000","last_fullsized":"2024-01-21T22:52:33.539211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:57:42.881371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":296,"num_write_kb":4241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"25'166","reported_seq":200,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.341937+0000","last_change":"2024-01-21T22:25:38.276745+0000","last_active":"2024-01-21T22:52:41.341937+0000","last_peered":"2024-01-21T22:52:41.341937+0000","last_clean":"2024-01-21T22:52:41.341937+0000","last_became_active":"2024-01-21T22:25:38.276512+0000","last_became_peered":"2024-01-21T22:25:38.276512+0000","last_unstale":"2024-01-21T22:52:41.341937+0000","last_undegraded":"2024-01-21T22:52:41.341937+0000","last_fullsized":"2024-01-21T22:52:41.341937+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:12.153042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":119,"num_write_kb":11355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"25'627","reported_seq":655,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:03.142512+0000","last_change":"2024-01-21T22:25:03.708366+0000","last_active":"2024-01-21T22:52:03.142512+0000","last_peered":"2024-01-21T22:52:03.142512+0000","last_clean":"2024-01-21T22:52:03.142512+0000","last_became_active":"2024-01-21T22:25:03.708101+0000","last_became_peered":"2024-01-21T22:25:03.708101+0000","last_unstale":"2024-01-21T22:52:03.142512+0000","last_undegraded":"2024-01-21T22:52:03.142512+0000","last_fullsized":"2024-01-21T22:52:03.142512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:57:42.881371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":625,"num_write_kb":4435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"25'198","reported_seq":232,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.426758+0000","last_change":"2024-01-21T22:25:04.877993+0000","last_active":"2024-01-21T22:52:41.426758+0000","last_peered":"2024-01-21T22:52:41.426758+0000","last_clean":"2024-01-21T22:52:41.426758+0000","last_became_active":"2024-01-21T22:25:04.877661+0000","last_became_peered":"2024-01-21T22:25:04.877661+0000","last_unstale":"2024-01-21T22:52:41.426758+0000","last_undegraded":"2024-01-21T22:52:41.426758+0000","last_fullsized":"2024-01-21T22:52:41.426758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:20:40.947205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":145,"num_write_kb":7036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"25'2","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.462672+0000","last_change":"2024-01-21T22:25:03.721559+0000","last_active":"2024-01-21T22:52:28.462672+0000","last_peered":"2024-01-21T22:52:28.462672+0000","last_clean":"2024-01-21T22:52:28.462672+0000","last_became_active":"2024-01-21T22:25:03.721281+0000","last_became_peered":"2024-01-21T22:25:03.721281+0000","last_unstale":"2024-01-21T22:52:28.462672+0000","last_undegraded":"2024-01-21T22:52:28.462672+0000","last_fullsized":"2024-01-21T22:52:28.462672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:55:19.679388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"25'173","reported_seq":217,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.408217+0000","last_change":"2024-01-21T22:25:04.883087+0000","last_active":"2024-01-21T22:52:41.408217+0000","last_peered":"2024-01-21T22:52:41.408217+0000","last_clean":"2024-01-21T22:52:41.408217+0000","last_became_active":"2024-01-21T22:25:04.880227+0000","last_became_peered":"2024-01-21T22:25:04.880227+0000","last_unstale":"2024-01-21T22:52:41.408217+0000","last_undegraded":"2024-01-21T22:52:41.408217+0000","last_fullsized":"2024-01-21T22:52:41.408217+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:58:09.653538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":4318,"num_write":118,"num_write_kb":4353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":181,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:48:53.349412+0000","last_change":"2024-01-21T22:24:52.968324+0000","last_active":"2024-01-21T22:48:53.349412+0000","last_peered":"2024-01-21T22:48:53.349412+0000","last_clean":"2024-01-21T22:48:53.349412+0000","last_became_active":"2024-01-21T22:24:52.967018+0000","last_became_peered":"2024-01-21T22:24:52.967018+0000","last_unstale":"2024-01-21T22:48:53.349412+0000","last_undegraded":"2024-01-21T22:48:53.349412+0000","last_fullsized":"2024-01-21T22:48:53.349412+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:24:50.312540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:24:50.312540+0000","last_clean_scrub_stamp":"2024-01-21T22:24:50.312540+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:52:04.800073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"25'6","reported_seq":35,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:39.512305+0000","last_change":"2024-01-21T22:25:03.707407+0000","last_active":"2024-01-21T22:52:39.512305+0000","last_peered":"2024-01-21T22:52:39.512305+0000","last_clean":"2024-01-21T22:52:39.512305+0000","last_became_active":"2024-01-21T22:25:03.707085+0000","last_became_peered":"2024-01-21T22:25:03.707085+0000","last_unstale":"2024-01-21T22:52:39.512305+0000","last_undegraded":"2024-01-21T22:52:39.512305+0000","last_fullsized":"2024-01-21T22:52:39.512305+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:44:22.501512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"25'177","reported_seq":215,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.444439+0000","last_change":"2024-01-21T22:25:04.868564+0000","last_active":"2024-01-21T22:52:41.444439+0000","last_peered":"2024-01-21T22:52:41.444439+0000","last_clean":"2024-01-21T22:52:41.444439+0000","last_became_active":"2024-01-21T22:25:04.868247+0000","last_became_peered":"2024-01-21T22:25:04.868247+0000","last_unstale":"2024-01-21T22:52:41.444439+0000","last_undegraded":"2024-01-21T22:52:41.444439+0000","last_fullsized":"2024-01-21T22:52:41.444439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:30:50.039501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2540,"num_write":132,"num_write_kb":4885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"25'1293","reported_seq":1322,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.530831+0000","last_change":"2024-01-21T22:25:03.706070+0000","last_active":"2024-01-21T22:52:33.530831+0000","last_peered":"2024-01-21T22:52:33.530831+0000","last_clean":"2024-01-21T22:52:33.530831+0000","last_became_active":"2024-01-21T22:25:03.705785+0000","last_became_peered":"2024-01-21T22:25:03.705785+0000","last_unstale":"2024-01-21T22:52:33.530831+0000","last_undegraded":"2024-01-21T22:52:33.530831+0000","last_fullsized":"2024-01-21T22:52:33.530831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1293,"log_dups_size":0,"ondisk_log_size":1293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:30.929444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1286,"num_write_kb":8869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"25'188","reported_seq":228,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.348728+0000","last_change":"2024-01-21T22:25:04.845436+0000","last_active":"2024-01-21T22:52:41.348728+0000","last_peered":"2024-01-21T22:52:41.348728+0000","last_clean":"2024-01-21T22:52:41.348728+0000","last_became_active":"2024-01-21T22:25:04.845274+0000","last_became_peered":"2024-01-21T22:25:04.845274+0000","last_unstale":"2024-01-21T22:52:41.348728+0000","last_undegraded":"2024-01-21T22:52:41.348728+0000","last_fullsized":"2024-01-21T22:52:41.348728+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:08:36.686787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":151,"num_write_kb":10622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"25'756","reported_seq":1531,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:39.468646+0000","last_change":"2024-01-21T22:25:03.709174+0000","last_active":"2024-01-21T22:52:39.468646+0000","last_peered":"2024-01-21T22:52:39.468646+0000","last_clean":"2024-01-21T22:52:39.468646+0000","last_became_active":"2024-01-21T22:25:03.708894+0000","last_became_peered":"2024-01-21T22:25:03.708894+0000","last_unstale":"2024-01-21T22:52:39.468646+0000","last_undegraded":"2024-01-21T22:52:39.468646+0000","last_fullsized":"2024-01-21T22:52:39.468646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:19:26.591140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":610747,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":746,"num_read_kb":1116,"num_write":750,"num_write_kb":1119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"25'188","reported_seq":230,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.418623+0000","last_change":"2024-01-21T22:25:04.711947+0000","last_active":"2024-01-21T22:52:41.418623+0000","last_peered":"2024-01-21T22:52:41.418623+0000","last_clean":"2024-01-21T22:52:41.418623+0000","last_became_active":"2024-01-21T22:25:04.711694+0000","last_became_peered":"2024-01-21T22:25:04.711694+0000","last_unstale":"2024-01-21T22:52:41.418623+0000","last_undegraded":"2024-01-21T22:52:41.418623+0000","last_fullsized":"2024-01-21T22:52:41.418623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:37:57.817795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3556,"num_write":135,"num_write_kb":5944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"25'1296","reported_seq":1325,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:39.513740+0000","last_change":"2024-01-21T22:25:03.706355+0000","last_active":"2024-01-21T22:52:39.513740+0000","last_peered":"2024-01-21T22:52:39.513740+0000","last_clean":"2024-01-21T22:52:39.513740+0000","last_became_active":"2024-01-21T22:25:03.703876+0000","last_became_peered":"2024-01-21T22:25:03.703876+0000","last_unstale":"2024-01-21T22:52:39.513740+0000","last_undegraded":"2024-01-21T22:52:39.513740+0000","last_fullsized":"2024-01-21T22:52:39.513740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1296,"log_dups_size":0,"ondisk_log_size":1296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:42:38.729006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1291,"num_write_kb":8882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"25'230","reported_seq":267,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.422692+0000","last_change":"2024-01-21T22:25:04.885997+0000","last_active":"2024-01-21T22:52:41.422692+0000","last_peered":"2024-01-21T22:52:41.422692+0000","last_clean":"2024-01-21T22:52:41.422692+0000","last_became_active":"2024-01-21T22:25:04.884162+0000","last_became_peered":"2024-01-21T22:25:04.884162+0000","last_unstale":"2024-01-21T22:52:41.422692+0000","last_undegraded":"2024-01-21T22:52:41.422692+0000","last_fullsized":"2024-01-21T22:52:41.422692+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:44:55.158990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":180,"num_write_kb":15842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"25'4","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.542371+0000","last_change":"2024-01-21T22:25:03.708300+0000","last_active":"2024-01-21T22:52:33.542371+0000","last_peered":"2024-01-21T22:52:33.542371+0000","last_clean":"2024-01-21T22:52:33.542371+0000","last_became_active":"2024-01-21T22:25:03.708036+0000","last_became_peered":"2024-01-21T22:25:03.708036+0000","last_unstale":"2024-01-21T22:52:33.542371+0000","last_undegraded":"2024-01-21T22:52:33.542371+0000","last_fullsized":"2024-01-21T22:52:33.542371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:23:51.196159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"25'188","reported_seq":232,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.352498+0000","last_change":"2024-01-21T22:25:04.844713+0000","last_active":"2024-01-21T22:52:41.352498+0000","last_peered":"2024-01-21T22:52:41.352498+0000","last_clean":"2024-01-21T22:52:41.352498+0000","last_became_active":"2024-01-21T22:25:04.844008+0000","last_became_peered":"2024-01-21T22:25:04.844008+0000","last_unstale":"2024-01-21T22:52:41.352498+0000","last_undegraded":"2024-01-21T22:52:41.352498+0000","last_fullsized":"2024-01-21T22:52:41.352498+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:33:25.760064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":4064,"num_write":142,"num_write_kb":4003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"25'3","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.529957+0000","last_change":"2024-01-21T22:25:03.699093+0000","last_active":"2024-01-21T22:52:33.529957+0000","last_peered":"2024-01-21T22:52:33.529957+0000","last_clean":"2024-01-21T22:52:33.529957+0000","last_became_active":"2024-01-21T22:25:03.698443+0000","last_became_peered":"2024-01-21T22:25:03.698443+0000","last_unstale":"2024-01-21T22:52:33.529957+0000","last_undegraded":"2024-01-21T22:52:33.529957+0000","last_fullsized":"2024-01-21T22:52:33.529957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:20:39.699087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"25'213","reported_seq":248,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.343518+0000","last_change":"2024-01-21T22:25:04.842733+0000","last_active":"2024-01-21T22:52:41.343518+0000","last_peered":"2024-01-21T22:52:41.343518+0000","last_clean":"2024-01-21T22:52:41.343518+0000","last_became_active":"2024-01-21T22:25:04.842479+0000","last_became_peered":"2024-01-21T22:25:04.842479+0000","last_unstale":"2024-01-21T22:52:41.343518+0000","last_undegraded":"2024-01-21T22:52:41.343518+0000","last_fullsized":"2024-01-21T22:52:41.343518+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:51:06.931790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":156,"num_write_kb":5205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"25'1421","reported_seq":1449,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:40.162054+0000","last_change":"2024-01-21T22:25:03.720306+0000","last_active":"2024-01-21T22:52:40.162054+0000","last_peered":"2024-01-21T22:52:40.162054+0000","last_clean":"2024-01-21T22:52:40.162054+0000","last_became_active":"2024-01-21T22:25:03.720067+0000","last_became_peered":"2024-01-21T22:25:03.720067+0000","last_unstale":"2024-01-21T22:52:40.162054+0000","last_undegraded":"2024-01-21T22:52:40.162054+0000","last_fullsized":"2024-01-21T22:52:40.162054+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1421,"log_dups_size":0,"ondisk_log_size":1421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:52:22.985590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3488,"num_write_kb":1512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"25'225","reported_seq":275,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.444231+0000","last_change":"2024-01-21T22:25:04.859628+0000","last_active":"2024-01-21T22:52:41.444231+0000","last_peered":"2024-01-21T22:52:41.444231+0000","last_clean":"2024-01-21T22:52:41.444231+0000","last_became_active":"2024-01-21T22:25:04.859270+0000","last_became_peered":"2024-01-21T22:25:04.859270+0000","last_unstale":"2024-01-21T22:52:41.444231+0000","last_undegraded":"2024-01-21T22:52:41.444231+0000","last_fullsized":"2024-01-21T22:52:41.444231+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:56:24.065539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":5842,"num_write":177,"num_write_kb":7531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"25'7","reported_seq":35,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.532457+0000","last_change":"2024-01-21T22:25:03.722006+0000","last_active":"2024-01-21T22:52:33.532457+0000","last_peered":"2024-01-21T22:52:33.532457+0000","last_clean":"2024-01-21T22:52:33.532457+0000","last_became_active":"2024-01-21T22:25:03.721275+0000","last_became_peered":"2024-01-21T22:25:03.721275+0000","last_unstale":"2024-01-21T22:52:33.532457+0000","last_undegraded":"2024-01-21T22:52:33.532457+0000","last_fullsized":"2024-01-21T22:52:33.532457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:43:53.801587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"25'182","reported_seq":222,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.445471+0000","last_change":"2024-01-21T22:25:04.858864+0000","last_active":"2024-01-21T22:52:41.445471+0000","last_peered":"2024-01-21T22:52:41.445471+0000","last_clean":"2024-01-21T22:52:41.445471+0000","last_became_active":"2024-01-21T22:25:04.858586+0000","last_became_peered":"2024-01-21T22:25:04.858586+0000","last_unstale":"2024-01-21T22:52:41.445471+0000","last_undegraded":"2024-01-21T22:52:41.445471+0000","last_fullsized":"2024-01-21T22:52:41.445471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:32.177562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":132,"num_write_kb":7745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"25'671","reported_seq":699,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.533281+0000","last_change":"2024-01-21T22:25:03.718436+0000","last_active":"2024-01-21T22:52:33.533281+0000","last_peered":"2024-01-21T22:52:33.533281+0000","last_clean":"2024-01-21T22:52:33.533281+0000","last_became_active":"2024-01-21T22:25:03.718171+0000","last_became_peered":"2024-01-21T22:25:03.718171+0000","last_unstale":"2024-01-21T22:52:33.533281+0000","last_undegraded":"2024-01-21T22:52:33.533281+0000","last_fullsized":"2024-01-21T22:52:33.533281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:05:40.702113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":666,"num_write_kb":4450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"25'192","reported_seq":228,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:19.939433+0000","last_change":"2024-01-21T22:25:04.875736+0000","last_active":"2024-01-21T22:52:19.939433+0000","last_peered":"2024-01-21T22:52:19.939433+0000","last_clean":"2024-01-21T22:52:19.939433+0000","last_became_active":"2024-01-21T22:25:04.875368+0000","last_became_peered":"2024-01-21T22:25:04.875368+0000","last_unstale":"2024-01-21T22:52:19.939433+0000","last_undegraded":"2024-01-21T22:52:19.939433+0000","last_fullsized":"2024-01-21T22:52:19.939433+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:39:31.273250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":138,"num_write_kb":6957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"25'4","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.539666+0000","last_change":"2024-01-21T22:25:03.722820+0000","last_active":"2024-01-21T22:52:33.539666+0000","last_peered":"2024-01-21T22:52:33.539666+0000","last_clean":"2024-01-21T22:52:33.539666+0000","last_became_active":"2024-01-21T22:25:03.722525+0000","last_became_peered":"2024-01-21T22:25:03.722525+0000","last_unstale":"2024-01-21T22:52:33.539666+0000","last_undegraded":"2024-01-21T22:52:33.539666+0000","last_fullsized":"2024-01-21T22:52:33.539666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:40:04.306585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"25'178","reported_seq":218,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.444594+0000","last_change":"2024-01-21T22:25:04.857451+0000","last_active":"2024-01-21T22:52:41.444594+0000","last_peered":"2024-01-21T22:52:41.444594+0000","last_clean":"2024-01-21T22:52:41.444594+0000","last_became_active":"2024-01-21T22:25:04.857205+0000","last_became_peered":"2024-01-21T22:25:04.857205+0000","last_unstale":"2024-01-21T22:52:41.444594+0000","last_undegraded":"2024-01-21T22:52:41.444594+0000","last_fullsized":"2024-01-21T22:52:41.444594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:12.153042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":126,"num_write_kb":4840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"25'2","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:51:08.155593+0000","last_change":"2024-01-21T22:25:03.708574+0000","last_active":"2024-01-21T22:51:08.155593+0000","last_peered":"2024-01-21T22:51:08.155593+0000","last_clean":"2024-01-21T22:51:08.155593+0000","last_became_active":"2024-01-21T22:25:03.708318+0000","last_became_peered":"2024-01-21T22:25:03.708318+0000","last_unstale":"2024-01-21T22:51:08.155593+0000","last_undegraded":"2024-01-21T22:51:08.155593+0000","last_fullsized":"2024-01-21T22:51:08.155593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:50:40.058463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"25'222","reported_seq":282,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.330361+0000","last_change":"2024-01-21T22:25:04.843079+0000","last_active":"2024-01-21T22:52:41.330361+0000","last_peered":"2024-01-21T22:52:41.330361+0000","last_clean":"2024-01-21T22:52:41.330361+0000","last_became_active":"2024-01-21T22:25:04.842815+0000","last_became_peered":"2024-01-21T22:25:04.842815+0000","last_unstale":"2024-01-21T22:52:41.330361+0000","last_undegraded":"2024-01-21T22:52:41.330361+0000","last_fullsized":"2024-01-21T22:52:41.330361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:16:41.440927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":8382,"num_write":174,"num_write_kb":9170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"25'764","reported_seq":792,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.533058+0000","last_change":"2024-01-21T22:25:03.710506+0000","last_active":"2024-01-21T22:52:33.533058+0000","last_peered":"2024-01-21T22:52:33.533058+0000","last_clean":"2024-01-21T22:52:33.533058+0000","last_became_active":"2024-01-21T22:25:03.710262+0000","last_became_peered":"2024-01-21T22:25:03.710262+0000","last_unstale":"2024-01-21T22:52:33.533058+0000","last_undegraded":"2024-01-21T22:52:33.533058+0000","last_fullsized":"2024-01-21T22:52:33.533058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":764,"log_dups_size":0,"ondisk_log_size":764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:25:49.132306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":756,"num_write_kb":4574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"25'193","reported_seq":234,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.410503+0000","last_change":"2024-01-21T22:25:04.876469+0000","last_active":"2024-01-21T22:52:41.410503+0000","last_peered":"2024-01-21T22:52:41.410503+0000","last_clean":"2024-01-21T22:52:41.410503+0000","last_became_active":"2024-01-21T22:25:04.875619+0000","last_became_peered":"2024-01-21T22:25:04.875619+0000","last_unstale":"2024-01-21T22:52:41.410503+0000","last_undegraded":"2024-01-21T22:52:41.410503+0000","last_fullsized":"2024-01-21T22:52:41.410503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:30:20.167607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3556,"num_write":140,"num_write_kb":4746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"25'155","reported_seq":188,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.405856+0000","last_change":"2024-01-21T22:25:04.891067+0000","last_active":"2024-01-21T22:52:41.405856+0000","last_peered":"2024-01-21T22:52:41.405856+0000","last_clean":"2024-01-21T22:52:41.405856+0000","last_became_active":"2024-01-21T22:25:04.890515+0000","last_became_peered":"2024-01-21T22:25:04.890515+0000","last_unstale":"2024-01-21T22:52:41.405856+0000","last_undegraded":"2024-01-21T22:52:41.405856+0000","last_fullsized":"2024-01-21T22:52:41.405856+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:25:48.726591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":88,"num_write_kb":3797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"25'205","reported_seq":232,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.151175+0000","last_change":"2024-01-21T22:25:03.719412+0000","last_active":"2024-01-21T22:52:41.151175+0000","last_peered":"2024-01-21T22:52:41.151175+0000","last_clean":"2024-01-21T22:52:41.151175+0000","last_became_active":"2024-01-21T22:25:03.718163+0000","last_became_peered":"2024-01-21T22:25:03.718163+0000","last_unstale":"2024-01-21T22:52:41.151175+0000","last_undegraded":"2024-01-21T22:52:41.151175+0000","last_fullsized":"2024-01-21T22:52:41.151175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:56:22.817420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1916505,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":203,"num_write_kb":1995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"25'212","reported_seq":244,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.446205+0000","last_change":"2024-01-21T22:25:04.889166+0000","last_active":"2024-01-21T22:52:41.446205+0000","last_peered":"2024-01-21T22:52:41.446205+0000","last_clean":"2024-01-21T22:52:41.446205+0000","last_became_active":"2024-01-21T22:25:04.888910+0000","last_became_peered":"2024-01-21T22:25:04.888910+0000","last_unstale":"2024-01-21T22:52:41.446205+0000","last_undegraded":"2024-01-21T22:52:41.446205+0000","last_fullsized":"2024-01-21T22:52:41.446205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:55:53.892439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":155,"num_write_kb":13062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"25'1484","reported_seq":1511,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.469678+0000","last_change":"2024-01-21T22:25:03.718932+0000","last_active":"2024-01-21T22:52:33.469678+0000","last_peered":"2024-01-21T22:52:33.469678+0000","last_clean":"2024-01-21T22:52:33.469678+0000","last_became_active":"2024-01-21T22:25:03.718649+0000","last_became_peered":"2024-01-21T22:25:03.718649+0000","last_unstale":"2024-01-21T22:52:33.469678+0000","last_undegraded":"2024-01-21T22:52:33.469678+0000","last_fullsized":"2024-01-21T22:52:33.469678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1484,"log_dups_size":0,"ondisk_log_size":1484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:26:30.929444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1478,"num_write_kb":9119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"25'210","reported_seq":265,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.419489+0000","last_change":"2024-01-21T22:25:04.715429+0000","last_active":"2024-01-21T22:52:41.419489+0000","last_peered":"2024-01-21T22:52:41.419489+0000","last_clean":"2024-01-21T22:52:41.419489+0000","last_became_active":"2024-01-21T22:25:04.714720+0000","last_became_peered":"2024-01-21T22:25:04.714720+0000","last_unstale":"2024-01-21T22:52:41.419489+0000","last_undegraded":"2024-01-21T22:52:41.419489+0000","last_fullsized":"2024-01-21T22:52:41.419489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:44:23.749631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":6858,"num_write":147,"num_write_kb":17332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"25'747","reported_seq":776,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.457894+0000","last_change":"2024-01-21T22:25:03.703289+0000","last_active":"2024-01-21T22:52:28.457894+0000","last_peered":"2024-01-21T22:52:28.457894+0000","last_clean":"2024-01-21T22:52:28.457894+0000","last_became_active":"2024-01-21T22:25:03.703020+0000","last_became_peered":"2024-01-21T22:25:03.703020+0000","last_unstale":"2024-01-21T22:52:28.457894+0000","last_undegraded":"2024-01-21T22:52:28.457894+0000","last_fullsized":"2024-01-21T22:52:28.457894+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:10.904923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":743,"num_write_kb":4567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"25'216","reported_seq":266,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.415285+0000","last_change":"2024-01-21T22:25:04.715894+0000","last_active":"2024-01-21T22:52:41.415285+0000","last_peered":"2024-01-21T22:52:41.415285+0000","last_clean":"2024-01-21T22:52:41.415285+0000","last_became_active":"2024-01-21T22:25:04.715652+0000","last_became_peered":"2024-01-21T22:25:04.715652+0000","last_unstale":"2024-01-21T22:52:41.415285+0000","last_undegraded":"2024-01-21T22:52:41.415285+0000","last_fullsized":"2024-01-21T22:52:41.415285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:13:04.601911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":5588,"num_write":182,"num_write_kb":13243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"25'762","reported_seq":791,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:23.498304+0000","last_change":"2024-01-21T22:25:03.702339+0000","last_active":"2024-01-21T22:52:23.498304+0000","last_peered":"2024-01-21T22:52:23.498304+0000","last_clean":"2024-01-21T22:52:23.498304+0000","last_became_active":"2024-01-21T22:25:03.702087+0000","last_became_peered":"2024-01-21T22:25:03.702087+0000","last_unstale":"2024-01-21T22:52:23.498304+0000","last_undegraded":"2024-01-21T22:52:23.498304+0000","last_fullsized":"2024-01-21T22:52:23.498304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":762,"log_dups_size":0,"ondisk_log_size":762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:51:05.683671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":760,"num_write_kb":4560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":833,"num_read_kb":211582,"num_write":8969,"num_write_kb":506711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12284,"ondisk_log_size":12284,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":82220498,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":746,"num_read_kb":1116,"num_write":17695,"num_write_kb":90278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164503552,"data_stored":164440996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15752,"ondisk_log_size":15752,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542484,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62376,"kb_used_data":13152,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94309464,"statfs":{"total":96636764160,"available":96572891136,"internally_reserved":0,"allocated":13467648,"data_stored":13290724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67700000000000005,"15min":0.68500000000000005},"min":{"1min":0.46000000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89600000000000002,"5min":1.0189999999999999,"15min":1.0640000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70699999999999996,"15min":0.69399999999999995},"min":{"1min":0.497,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.081,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.82399999999999995}]},{"osd":1,"last update":"Sun Jan 21 22:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71599999999999997,"15min":0.71099999999999997},"min":{"1min":0.55000000000000004,"5min":0.51700000000000002,"15min":0.34000000000000002},"max":{"1min":0.96899999999999997,"5min":1.159,"15min":1.4179999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.73399999999999999,"15min":0.72299999999999998},"min":{"1min":0.56000000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.95599999999999996,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.94399999999999995}]},{"osd":2,"last update":"Sun Jan 21 22:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69299999999999995,"15min":0.70099999999999996},"min":{"1min":0.49399999999999999,"5min":0.46000000000000002,"15min":0.39300000000000002},"max":{"1min":0.94699999999999995,"5min":1.0069999999999999,"15min":1.052},"last":1.002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73899999999999999,"15min":0.72499999999999998},"min":{"1min":0.58099999999999996,"5min":0.438,"15min":0.38},"max":{"1min":1.02,"5min":1.1140000000000001,"15min":1.1799999999999999},"last":0.71699999999999997}]},{"osd":3,"last update":"Sun Jan 21 22:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71099999999999997,"15min":0.71399999999999997},"min":{"1min":0.54500000000000004,"5min":0.42699999999999999,"15min":0.42099999999999999},"max":{"1min":1.048,"5min":1.093,"15min":1.1539999999999999},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.73199999999999998,"15min":0.74099999999999999},"min":{"1min":0.56299999999999994,"5min":0.48399999999999999,"15min":0.45600000000000002},"max":{"1min":1.117,"5min":1.117,"15min":1.181},"last":0.76300000000000001}]},{"osd":4,"last update":"Sun Jan 21 22:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.35099999999999998,"15min":0.34499999999999997},"min":{"1min":0.26600000000000001,"5min":0.245,"15min":0.22700000000000001},"max":{"1min":0.39800000000000002,"5min":0.57099999999999995,"15min":0.58299999999999996},"last":0.505},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.35199999999999998,"15min":0.34699999999999998},"min":{"1min":0.27300000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.41699999999999998,"5min":0.66700000000000004,"15min":0.68100000000000005},"last":0.51800000000000002}]},{"osd":5,"last update":"Sun Jan 21 22:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34300000000000003,"15min":0.34200000000000003},"min":{"1min":0.255,"5min":0.255,"15min":0.249},"max":{"1min":0.44500000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38100000000000001,"15min":0.372},"min":{"1min":0.31,"5min":0.26700000000000002,"15min":0.249},"max":{"1min":0.48599999999999999,"5min":0.54100000000000004,"15min":0.624},"last":0.56000000000000005}]},{"osd":6,"last update":"Sun Jan 21 22:51:48 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.40000000000000002,"15min":0.39500000000000002},"min":{"1min":0.33400000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.45200000000000001,"5min":0.57199999999999995,"15min":0.67300000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.38800000000000001,"15min":0.38600000000000001},"min":{"1min":0.313,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.52300000000000002,"5min":0.60699999999999998,"15min":0.63700000000000001},"last":0.55000000000000004}]}]},{"osd":6,"up_from":14,"seq":60129542484,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43720,"kb_used_data":12544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328120,"statfs":{"total":96636764160,"available":96591994880,"internally_reserved":0,"allocated":12845056,"data_stored":12679853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69299999999999995,"15min":0.70099999999999996},"min":{"1min":0.497,"5min":0.48299999999999998,"15min":0.45600000000000002},"max":{"1min":0.89200000000000002,"5min":1.0189999999999999,"15min":3.3380000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.54100000000000004,"5min":0.495,"15min":0.44600000000000001},"max":{"1min":0.93799999999999994,"5min":1.038,"15min":1.0669999999999999},"last":0.59399999999999997}]},{"osd":1,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72099999999999997,"15min":0.70699999999999996},"min":{"1min":0.46500000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0529999999999999,"5min":1.0860000000000001,"15min":2.9860000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72299999999999998,"15min":0.72099999999999997},"min":{"1min":0.52900000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.1020000000000001},"last":0.94999999999999996}]},{"osd":2,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.49299999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.88200000000000001,"5min":1.1220000000000001,"15min":7.5439999999999996},"last":0.749},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71999999999999997,"15min":0.73199999999999998},"min":{"1min":0.59499999999999997,"5min":0.56200000000000006,"15min":0.495},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.629}]},{"osd":3,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72399999999999998,"15min":0.70499999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.42599999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73099999999999998,"15min":0.78100000000000003},"min":{"1min":0.56399999999999995,"5min":0.52700000000000002,"15min":0.39400000000000002},"max":{"1min":0.90100000000000002,"5min":1.075,"15min":8.4689999999999994},"last":0.77000000000000002}]},{"osd":4,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34300000000000003,"15min":0.33800000000000002},"min":{"1min":0.26000000000000001,"5min":0.245,"15min":0.222},"max":{"1min":0.45200000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.35599999999999998,"15min":0.35199999999999998},"min":{"1min":0.24199999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.45100000000000001,"5min":0.45600000000000002,"15min":0.91900000000000004},"last":0.38600000000000001}]},{"osd":5,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35199999999999998,"15min":0.35399999999999998},"min":{"1min":0.30199999999999999,"5min":0.26400000000000001,"15min":0.26100000000000001},"max":{"1min":0.40200000000000002,"5min":0.53800000000000003,"15min":0.61299999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.377,"15min":0.36799999999999999},"min":{"1min":0.29599999999999999,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.45300000000000001,"5min":0.46800000000000003,"15min":0.52200000000000002},"last":0.371}]},{"osd":7,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.38},"min":{"1min":0.29799999999999999,"5min":0.27100000000000002,"15min":0.26600000000000001},"max":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.96099999999999997},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.39200000000000002,"15min":0.39100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28799999999999998,"15min":0.27300000000000002},"max":{"1min":0.45200000000000001,"5min":0.54900000000000004,"15min":0.98699999999999999},"last":0.42699999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542484,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":101612,"kb_used_data":26020,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94270228,"statfs":{"total":96636764160,"available":96532713472,"internally_reserved":0,"allocated":26644480,"data_stored":26463972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.68400000000000005,"15min":0.68899999999999995},"min":{"1min":0.46300000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.624},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.69999999999999996,"15min":0.69799999999999995},"min":{"1min":0.499,"5min":0.432,"15min":0.41499999999999998},"max":{"1min":0.83799999999999997,"5min":1.0149999999999999,"15min":1.1020000000000001},"last":0.89800000000000002}]},{"osd":1,"last update":"Sun Jan 21 22:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.67200000000000004,"15min":0.68999999999999995},"min":{"1min":0.56599999999999995,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91500000000000004,"5min":0.95599999999999996,"15min":1.1950000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.74099999999999999,"15min":0.72799999999999998},"min":{"1min":0.50800000000000001,"5min":0.49199999999999999,"15min":0.40999999999999998},"max":{"1min":0.96999999999999997,"5min":1.111,"15min":1.151},"last":0.59899999999999998}]},{"osd":2,"last update":"Sun Jan 21 22:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.72099999999999997,"15min":0.71899999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.434},"max":{"1min":1.046,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70399999999999996,"15min":0.71499999999999997},"min":{"1min":0.53300000000000003,"5min":0.437,"15min":0.437},"max":{"1min":0.85399999999999998,"5min":1.143,"15min":1.149},"last":0.57799999999999996}]},{"osd":3,"last update":"Sun Jan 21 22:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.70799999999999996,"15min":0.73399999999999999},"min":{"1min":0.48499999999999999,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.97499999999999998,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.749,"15min":0.71799999999999997},"min":{"1min":0.58799999999999997,"5min":0.46200000000000002,"15min":0.373},"max":{"1min":0.97199999999999998,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.91000000000000003}]},{"osd":4,"last update":"Sun Jan 21 22:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.32800000000000001,"15min":0.33400000000000002},"min":{"1min":0.254,"5min":0.23599999999999999,"15min":0.22800000000000001},"max":{"1min":0.433,"5min":0.46300000000000002,"15min":0.47099999999999997},"last":0.379},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.34399999999999997,"15min":0.34999999999999998},"min":{"1min":0.28100000000000003,"5min":0.26500000000000001,"15min":0.254},"max":{"1min":0.41199999999999998,"5min":0.50700000000000001,"15min":0.55400000000000005},"last":0.311}]},{"osd":6,"last update":"Sun Jan 21 22:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.27700000000000002,"5min":0.26300000000000001,"15min":0.26000000000000001},"max":{"1min":0.38600000000000001,"5min":0.48899999999999999,"15min":0.51800000000000002},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.25600000000000001},"max":{"1min":0.41699999999999998,"5min":0.48099999999999998,"15min":0.51100000000000001},"last":0.34999999999999998}]},{"osd":7,"last update":"Sun Jan 21 22:51:42 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.38,"15min":0.38200000000000001},"min":{"1min":0.28499999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.441,"5min":0.47799999999999998,"15min":0.64000000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.378,"5min":0.38200000000000001,"15min":0.38400000000000001},"min":{"1min":0.28199999999999997,"5min":0.27600000000000002,"15min":0.27100000000000002},"max":{"1min":0.45300000000000001,"5min":0.54100000000000004,"15min":0.57099999999999995},"last":0.36699999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542484,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65944,"kb_used_data":12560,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94305896,"statfs":{"total":96636764160,"available":96569237504,"internally_reserved":0,"allocated":12861440,"data_stored":12681197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.51100000000000001,"5min":0.46200000000000002,"15min":0.38800000000000001},"max":{"1min":0.97299999999999998,"5min":1.05,"15min":1.0880000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69399999999999995,"15min":0.71499999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.41099999999999998},"max":{"1min":0.998,"5min":1.008,"15min":2.9649999999999999},"last":0.51000000000000001}]},{"osd":1,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.70599999999999996,"15min":0.71899999999999997},"min":{"1min":0.45800000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.9410000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.70199999999999996,"15min":0.71399999999999997},"min":{"1min":0.44,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.615},"last":0.96299999999999997}]},{"osd":2,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70499999999999996,"15min":0.71499999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.95799999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73499999999999999,"15min":0.71799999999999997},"min":{"1min":0.54700000000000004,"5min":0.505,"15min":0.40400000000000003},"max":{"1min":1.05,"5min":1.05,"15min":1.4710000000000001},"last":0.90800000000000003}]},{"osd":3,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69899999999999995,"15min":0.69799999999999995},"min":{"1min":0.55400000000000005,"5min":0.48499999999999999,"15min":0.43099999999999999},"max":{"1min":0.97499999999999998,"5min":1.0640000000000001,"15min":1.0720000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.72699999999999998,"15min":0.745},"min":{"1min":0.53800000000000003,"5min":0.52300000000000002,"15min":0.50900000000000001},"max":{"1min":0.92700000000000005,"5min":1.0600000000000001,"15min":1.1160000000000001},"last":0.66900000000000004}]},{"osd":5,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34499999999999997,"15min":0.34200000000000003},"min":{"1min":0.251,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.432,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.35099999999999998,"15min":0.34799999999999998},"min":{"1min":0.28000000000000003,"5min":0.252,"15min":0.249},"max":{"1min":0.42299999999999999,"5min":0.443,"15min":0.49399999999999999},"last":0.45300000000000001}]},{"osd":6,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34999999999999998,"15min":0.34699999999999998},"min":{"1min":0.28100000000000003,"5min":0.24299999999999999,"15min":0.23599999999999999},"max":{"1min":0.42099999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.311},{"interface":"front","average":{"1min":0.376,"5min":0.378,"15min":0.374},"min":{"1min":0.29199999999999998,"5min":0.27700000000000002,"15min":0.26400000000000001},"max":{"1min":0.46800000000000003,"5min":0.51500000000000001,"15min":0.54000000000000004},"last":0.40000000000000002}]},{"osd":7,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.27600000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.52000000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.495},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39600000000000002,"15min":0.39000000000000001},"min":{"1min":0.317,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.47699999999999998,"5min":0.626,"15min":0.626},"last":0.38}]}]},{"osd":3,"up_from":14,"seq":60129542485,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104396,"kb_used_data":37124,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94267444,"statfs":{"total":96636764160,"available":96529862656,"internally_reserved":0,"allocated":38014976,"data_stored":37846259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.093,"15min":1.0820000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.5150000000000001,"5min":3.9849999999999999,"15min":6.1879999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":1.302,"5min":1.2929999999999999,"15min":1.278},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":4.7430000000000003,"5min":8.6639999999999997,"15min":8.6639999999999997},"last":0.51000000000000001}]},{"osd":1,"last update":"Sun Jan 21 22:52:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.3109999999999999,"15min":1.26},"min":{"1min":0.77500000000000002,"5min":0.751,"15min":0.67000000000000004},"max":{"1min":3.8799999999999999,"5min":17.492999999999999,"15min":17.492999999999999},"last":1.0940000000000001},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.472,"15min":1.4139999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":5.7270000000000003,"5min":8.9629999999999992,"15min":8.9629999999999992},"last":0.54100000000000004}]},{"osd":2,"last update":"Sun Jan 21 22:52:41 2024","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.4330000000000001,"15min":1.377},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":6.2119999999999997,"5min":8.5530000000000008,"15min":8.5530000000000008},"last":0.56200000000000006},{"interface":"front","average":{"1min":1.484,"5min":1.581,"15min":1.5209999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":4.3049999999999997,"5min":17.664999999999999,"15min":17.664999999999999},"last":0.82799999999999996}]},{"osd":4,"last update":"Sun Jan 21 22:52:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.1879999999999999,"15min":1.1479999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":4.226,"5min":4.5099999999999998,"15min":4.5099999999999998},"last":0.69199999999999995},{"interface":"front","average":{"1min":1.25,"5min":1.2,"15min":1.1539999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":6.0110000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":0.61699999999999999}]},{"osd":5,"last update":"Sun Jan 21 22:52:41 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.216,"15min":1.2010000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":9.2370000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.369,"5min":1.3799999999999999,"15min":1.3029999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":4.343,"5min":8.7420000000000009,"15min":8.7420000000000009},"last":0.63200000000000001}]},{"osd":6,"last update":"Sun Jan 21 22:52:41 2024","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.373,"15min":1.3660000000000001},"min":{"1min":0.89800000000000002,"5min":0.874,"15min":0.85899999999999999},"max":{"1min":4.6230000000000002,"5min":4.6230000000000002,"15min":9.3719999999999999},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.423,"15min":1.3799999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":6.1109999999999998,"5min":6.1109999999999998,"15min":6.1109999999999998},"last":0.66200000000000003}]},{"osd":7,"last update":"Sun Jan 21 22:52:41 2024","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.5069999999999999,"15min":1.498},"min":{"1min":1.1339999999999999,"5min":0.97199999999999998,"15min":0.86099999999999999},"max":{"1min":4.5359999999999996,"5min":4.5359999999999996,"15min":9.4610000000000003},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.5900000000000001,"15min":1.5169999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":5.242,"5min":8.8719999999999999,"15min":8.8719999999999999},"last":0.58999999999999997}]}]},{"osd":2,"up_from":14,"seq":60129542483,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61776,"kb_used_data":12552,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94310064,"statfs":{"total":96636764160,"available":96573505536,"internally_reserved":0,"allocated":12853248,"data_stored":12680525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0509999999999999,"15min":1.04},"min":{"1min":0.79100000000000004,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5369999999999999,"5min":1.9410000000000001,"15min":2.0419999999999998},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.306,"15min":1.2769999999999999},"min":{"1min":0.95599999999999996,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.6299999999999999,"5min":2.2770000000000001,"15min":2.2770000000000001},"last":1.119}]},{"osd":1,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.29,"15min":1.2629999999999999},"min":{"1min":0.88400000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.605,"5min":1.9399999999999999,"15min":2.234},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.3540000000000001,"15min":1.3759999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.8540000000000001,"5min":2.411,"15min":2.411},"last":0.55700000000000005}]},{"osd":3,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.27,"15min":1.296},"min":{"1min":0.96799999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.556,"5min":1.788,"15min":1.9690000000000001},"last":1.073},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.429,"15min":1.4410000000000001},"min":{"1min":1.2230000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.681,"5min":2.0659999999999998,"15min":2.1320000000000001},"last":1.1579999999999999}]},{"osd":4,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.149,"15min":1.155},"min":{"1min":0.90100000000000002,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.345,"5min":1.776,"15min":2.0259999999999998},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0720000000000001,"15min":1.0820000000000001},"min":{"1min":0.69999999999999996,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4199999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.96399999999999997}]},{"osd":5,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.79900000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.415,"5min":1.623,"15min":1.669},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.244,"5min":1.2809999999999999,"15min":1.29},"min":{"1min":1.0860000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5720000000000001,"5min":1.8859999999999999,"15min":2.125},"last":1.0569999999999999}]},{"osd":6,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.244,"15min":1.25},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.6160000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.2130000000000001,"15min":1.254},"min":{"1min":0.96799999999999997,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.5880000000000001,"5min":1.8600000000000001,"15min":1.8919999999999999},"last":1.5980000000000001}]},{"osd":7,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.377,"15min":1.4019999999999999},"min":{"1min":1.034,"5min":0.748,"15min":0.748},"max":{"1min":1.7450000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3640000000000001,"15min":1.391},"min":{"1min":1.077,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.694,"5min":1.9139999999999999,"15min":2.004},"last":1.6919999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542484,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76576,"kb_used_data":23192,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94295264,"statfs":{"total":96636764160,"available":96558350336,"internally_reserved":0,"allocated":23748608,"data_stored":23575404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.0620000000000001,"15min":1.0509999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":2.3839999999999999,"5min":2.3839999999999999,"15min":3.613},"last":0.74399999999999999},{"interface":"front","average":{"1min":1.333,"5min":1.2430000000000001,"15min":1.214},"min":{"1min":0.76000000000000001,"5min":0.72499999999999998,"15min":0.63800000000000001},"max":{"1min":3.3879999999999999,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.66400000000000003}]},{"osd":2,"last update":"Sun Jan 21 22:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.248,"15min":1.236},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.73799999999999999},"max":{"1min":3.056,"5min":3.056,"15min":4.2309999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.355,"15min":1.353},"min":{"1min":0.996,"5min":0.82499999999999996,"15min":0.76700000000000002},"max":{"1min":2.8159999999999998,"5min":2.8159999999999998,"15min":3.2450000000000001},"last":1.105}]},{"osd":3,"last update":"Sun Jan 21 22:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.4339999999999999,"15min":1.425},"min":{"1min":1.0189999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":4.0890000000000004,"5min":4.0890000000000004,"15min":4.0890000000000004},"last":0.83999999999999997},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.3540000000000001,"15min":1.3560000000000001},"min":{"1min":0.874,"5min":0.84599999999999997,"15min":0.78100000000000003},"max":{"1min":3.8519999999999999,"5min":3.8519999999999999,"15min":3.8519999999999999},"last":1.1539999999999999}]},{"osd":4,"last update":"Sun Jan 21 22:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.0389999999999999,"5min":1.0660000000000001,"15min":1.028},"min":{"1min":0.67700000000000005,"5min":0.67200000000000004,"15min":0.65600000000000003},"max":{"1min":2.8639999999999999,"5min":2.8639999999999999,"15min":2.8639999999999999},"last":0.622},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.206,"15min":1.1879999999999999},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.74299999999999999},"max":{"1min":2.5939999999999999,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":0.91900000000000004}]},{"osd":5,"last update":"Sun Jan 21 22:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1140000000000001,"15min":1.1100000000000001},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71099999999999997},"max":{"1min":3.0139999999999998,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":0.878},{"interface":"front","average":{"1min":1.194,"5min":1.26,"15min":1.2410000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.972,"5min":2.972,"15min":2.972},"last":0.72199999999999998}]},{"osd":6,"last update":"Sun Jan 21 22:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.302,"15min":1.3009999999999999},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.79300000000000004},"max":{"1min":3.121,"5min":3.121,"15min":3.121},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.4279999999999999,"15min":1.399},"min":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.88200000000000001},"max":{"1min":3.3410000000000002,"5min":3.3410000000000002,"15min":3.3410000000000002},"last":0.79100000000000004}]},{"osd":7,"last update":"Sun Jan 21 22:51:45 2024","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.86799999999999999},"max":{"1min":2.718,"5min":2.718,"15min":2.718},"last":1.006},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.4410000000000001,"15min":1.4710000000000001},"min":{"1min":1.002,"5min":1.002,"15min":0.93200000000000005},"max":{"1min":3.218,"5min":3.218,"15min":3.218},"last":1.0609999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542485,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":93984,"kb_used_data":26712,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94277856,"statfs":{"total":96636764160,"available":96540524544,"internally_reserved":0,"allocated":27353088,"data_stored":27179326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0309999999999999,"15min":1.0640000000000001},"min":{"1min":0.70499999999999996,"5min":0.624,"15min":0.621},"max":{"1min":1.6080000000000001,"5min":1.6870000000000001,"15min":1.988},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.232,"15min":1.2889999999999999},"min":{"1min":0.90100000000000002,"5min":0.70199999999999996,"15min":0.67200000000000004},"max":{"1min":1.609,"5min":1.718,"15min":11.01},"last":1.153}]},{"osd":2,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.306,"15min":1.3300000000000001},"min":{"1min":1.2050000000000001,"5min":0.83799999999999997,"15min":0.81699999999999995},"max":{"1min":1.8580000000000001,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.3480000000000001,"15min":1.3879999999999999},"min":{"1min":0.96199999999999997,"5min":0.84499999999999997,"15min":0.84399999999999997},"max":{"1min":1.742,"5min":1.9690000000000001,"15min":11.308999999999999},"last":1.298}]},{"osd":3,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.268,"15min":1.3129999999999999},"min":{"1min":0.98999999999999999,"5min":0.85399999999999998,"15min":0.67000000000000004},"max":{"1min":1.573,"5min":1.671,"15min":9.2929999999999993},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.462,"5min":1.4390000000000001,"15min":1.4850000000000001},"min":{"1min":1.1679999999999999,"5min":1.01,"15min":0.91700000000000004},"max":{"1min":1.696,"5min":2.0390000000000001,"15min":11.4},"last":1.9399999999999999}]},{"osd":4,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0660000000000001,"15min":1.0860000000000001},"min":{"1min":0.82099999999999995,"5min":0.70999999999999996,"15min":0.66200000000000003},"max":{"1min":1.3200000000000001,"5min":1.593,"15min":2.0110000000000001},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.157,"15min":1.2050000000000001},"min":{"1min":0.94099999999999995,"5min":0.83899999999999997,"15min":0.77300000000000002},"max":{"1min":1.823,"5min":1.823,"15min":11.345000000000001},"last":1.819}]},{"osd":5,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.0289999999999999,"15min":1.0549999999999999},"min":{"1min":0.76200000000000001,"5min":0.71699999999999997,"15min":0.70299999999999996},"max":{"1min":1.498,"5min":1.7470000000000001,"15min":2.0030000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2190000000000001,"15min":1.266},"min":{"1min":0.84399999999999997,"5min":0.82799999999999996,"15min":0.66300000000000003},"max":{"1min":1.4099999999999999,"5min":1.6539999999999999,"15min":2.1509999999999998},"last":1.3759999999999999}]},{"osd":6,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1819999999999999,"15min":1.2130000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.369,"5min":1.871,"15min":2.198},"last":1.012},{"interface":"front","average":{"1min":1.353,"5min":1.3540000000000001,"15min":1.397},"min":{"1min":1.0269999999999999,"5min":0.91000000000000003,"15min":0.872},"max":{"1min":1.74,"5min":1.8120000000000001,"15min":2.2509999999999999},"last":1.4730000000000001}]},{"osd":7,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.48,"5min":1.3680000000000001,"15min":1.4219999999999999},"min":{"1min":1.1839999999999999,"5min":0.96299999999999997,"15min":0.85799999999999998},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":11.374000000000001},"last":1.8799999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3460000000000001,"15min":1.369},"min":{"1min":0.97899999999999998,"5min":0.94999999999999996,"15min":0.875},"max":{"1min":1.855,"5min":1.9810000000000001,"15min":2.3319999999999999},"last":1.2030000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":27090944,"data_stored":27082385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22892544,"data_stored":22888095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12591104,"data_stored":12583584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":37752832,"data_stored":37749318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12599296,"data_stored":12584256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":25788416,"data_stored":25776663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":13205504,"data_stored":13193783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-21T22:52:44.146 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-21T22:52:44.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.389+0000 7f993ea00640 1 Processor -- start 2024-01-21T22:52:44.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.390+0000 7f993ea00640 1 -- start start 2024-01-21T22:52:44.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.391+0000 7f993ea00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9938153a50 0x7f99380c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:44.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.391+0000 7f993ea00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f993806b1c0 0x7f993806b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:44.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.391+0000 7f993ea00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99380ca000 con 0x7f993806c0a0 2024-01-21T22:52:44.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.391+0000 7f993ea00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f993806ad30 con 0x7f9938153a50 2024-01-21T22:52:44.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.391+0000 7f993ea00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99380cb8d0 con 0x7f993806b1c0 2024-01-21T22:52:44.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.392+0000 7f993d600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9938153a50 0x7f99380c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:44.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.392+0000 7f993d600640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f9938153a50 0x7f99380c9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50914/0 (socket says 172.21.15.138:50914) 2024-01-21T22:52:44.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.392+0000 7f993d600640 1 -- 172.21.15.138:0/4263439513 learned_addr learned my addr 172.21.15.138:0/4263439513 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:44.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.395+0000 7f993d600640 1 -- 172.21.15.138:0/4263439513 >> v2:172.21.15.152:3300/0 conn(0x7f993806b1c0 msgr2=0x7f993806b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:44.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.395+0000 7f993d600640 1 --2- 172.21.15.138:0/4263439513 >> v2:172.21.15.152:3300/0 conn(0x7f993806b1c0 0x7f993806b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:44.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.395+0000 7f993d600640 1 -- 172.21.15.138:0/4263439513 >> v1:172.21.15.152:6789/0 conn(0x7f993806c0a0 legacy=0x7f99380cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:44.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.395+0000 7f993d600640 1 -- 172.21.15.138:0/4263439513 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993806bb70 con 0x7f9938153a50 2024-01-21T22:52:44.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.397+0000 7f993d600640 1 --2- 172.21.15.138:0/4263439513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9938153a50 0x7f99380c9ac0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f99380b1ff0 tx=0x7f992802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6e4229adbc670c server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:44.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.402+0000 7f9937e00640 1 -- 172.21.15.138:0/4263439513 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992802c9f0 con 0x7f9938153a50 2024-01-21T22:52:44.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.402+0000 7f9937e00640 1 -- 172.21.15.138:0/4263439513 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992802cb50 con 0x7f9938153a50 2024-01-21T22:52:44.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.402+0000 7f9937e00640 1 -- 172.21.15.138:0/4263439513 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99280343f0 con 0x7f9938153a50 2024-01-21T22:52:44.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.403+0000 7f993ea00640 1 -- 172.21.15.138:0/4263439513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9938153a50 msgr2=0x7f99380c9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:44.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.403+0000 7f993ea00640 1 --2- 172.21.15.138:0/4263439513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9938153a50 0x7f99380c9ac0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f99380b1ff0 tx=0x7f992802be50 comp rx=0 tx=0).stop 2024-01-21T22:52:44.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.403+0000 7f993ea00640 1 -- 172.21.15.138:0/4263439513 shutdown_connections 2024-01-21T22:52:44.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.403+0000 7f993ea00640 1 --2- 172.21.15.138:0/4263439513 >> v2:172.21.15.152:3300/0 conn(0x7f993806b1c0 0x7f993806b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:44.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.403+0000 7f993ea00640 1 --2- 172.21.15.138:0/4263439513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9938153a50 0x7f99380c9ac0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:44.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.403+0000 7f993ea00640 1 -- 172.21.15.138:0/4263439513 >> 172.21.15.138:0/4263439513 conn(0x7f993806d240 msgr2=0x7f9938066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:44.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.403+0000 7f993ea00640 1 -- 172.21.15.138:0/4263439513 shutdown_connections 2024-01-21T22:52:44.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.403+0000 7f993ea00640 1 -- 172.21.15.138:0/4263439513 wait complete. 2024-01-21T22:52:44.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.405+0000 7f993ea00640 1 Processor -- start 2024-01-21T22:52:44.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.405+0000 7f993ea00640 1 -- start start 2024-01-21T22:52:44.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.406+0000 7f993ea00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f993806b1c0 0x7f9938147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:44.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.406+0000 7f993ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f993806c0a0 0x7f993815bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:44.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.406+0000 7f993ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9938153a50 0x7f993815c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:44.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.406+0000 7f993ea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9938147750 con 0x7f9938153a50 2024-01-21T22:52:44.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.406+0000 7f993ea00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99380b3e00 con 0x7f993806c0a0 2024-01-21T22:52:44.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.406+0000 7f993ea00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99380b43c0 con 0x7f993806b1c0 2024-01-21T22:52:44.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.407+0000 7f993d600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f993806b1c0 0x7f9938147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:44.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.407+0000 7f993e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9938153a50 0x7f993815c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:44.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.408+0000 7f993d600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f993806b1c0 0x7f9938147210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:55238/0 (socket says 172.21.15.138:55238) 2024-01-21T22:52:44.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.408+0000 7f993d600640 1 -- 172.21.15.138:0/1171067332 learned_addr learned my addr 172.21.15.138:0/1171067332 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:44.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.411+0000 7f993d600640 1 -- 172.21.15.138:0/1171067332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f993806c0a0 msgr2=0x7f993815bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:44.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.411+0000 7f993d600640 1 --2- 172.21.15.138:0/1171067332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f993806c0a0 0x7f993815bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:44.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.411+0000 7f993d600640 1 -- 172.21.15.138:0/1171067332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9938153a50 msgr2=0x7f993815c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:44.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.411+0000 7f993d600640 1 --2- 172.21.15.138:0/1171067332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9938153a50 0x7f993815c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:44.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.411+0000 7f993d600640 1 -- 172.21.15.138:0/1171067332 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9928009b30 con 0x7f993806b1c0 2024-01-21T22:52:44.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.413+0000 7f993d600640 1 --2- 172.21.15.138:0/1171067332 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f993806b1c0 0x7f9938147210 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9928036040 tx=0x7f9928004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:44.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.417+0000 7f9935600640 1 -- 172.21.15.138:0/1171067332 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992803e450 con 0x7f993806b1c0 2024-01-21T22:52:44.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.418+0000 7f9935600640 1 -- 172.21.15.138:0/1171067332 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992803e5d0 con 0x7f993806b1c0 2024-01-21T22:52:44.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.418+0000 7f9935600640 1 -- 172.21.15.138:0/1171067332 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992803e980 con 0x7f993806b1c0 2024-01-21T22:52:44.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.418+0000 7f993ea00640 1 -- 172.21.15.138:0/1171067332 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9938154220 con 0x7f993806b1c0 2024-01-21T22:52:44.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.418+0000 7f993ea00640 1 -- 172.21.15.138:0/1171067332 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99380b4050 con 0x7f993806b1c0 2024-01-21T22:52:44.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.419+0000 7f993ea00640 1 -- 172.21.15.138:0/1171067332 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99381eee90 con 0x7f993806b1c0 2024-01-21T22:52:44.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.439+0000 7f9935600640 1 -- 172.21.15.138:0/1171067332 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f992803eae0 con 0x7f993806b1c0 2024-01-21T22:52:44.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.440+0000 7f9935600640 1 --2- 172.21.15.138:0/1171067332 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f991006b380 0x7f991006d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:44.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.440+0000 7f993cc00640 1 --2- 172.21.15.138:0/1171067332 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f991006b380 0x7f991006d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:44.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.441+0000 7f993cc00640 1 --2- 172.21.15.138:0/1171067332 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f991006b380 0x7f991006d840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f99380c5620 tx=0x7f9920007990 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:44.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.443+0000 7f9935600640 1 -- 172.21.15.138:0/1171067332 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f992807c010 con 0x7f993806b1c0 2024-01-21T22:52:44.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.749+0000 7f9935600640 1 -- 172.21.15.138:0/1171067332 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f9928077c50 con 0x7f993806b1c0 2024-01-21T22:52:44.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.947+0000 7f993ea00640 1 -- 172.21.15.138:0/1171067332 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99380ca000 con 0x7f993806b1c0 2024-01-21T22:52:44.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.987+0000 7f9935600640 1 -- 172.21.15.138:0/1171067332 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16570 (secure 0 0 0) 0x7f99380ca000 con 0x7f993806b1c0 2024-01-21T22:52:44.989 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:52:44.989 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":25,"fsid":"d488d446-149f-48d0-89ca-b2830ae72846","created":"2024-01-21T22:24:28.581932+0000","modified":"2024-01-21T22:25:37.558325+0000","last_up_change":"2024-01-21T22:24:47.145799+0000","last_in_change":"2024-01-21T22:24:37.672807+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-21T22:24:49.586178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-21T22:25:02.044454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-21T22:25:03.361765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d812b2d6-c8a3-4015-bfdb-5766b25eae5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6825","nonce":2849203277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6827","nonce":2849203277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6831","nonce":2849203277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6829","nonce":2849203277}]},"public_addr":"172.21.15.138:6825/2849203277","cluster_addr":"172.21.15.138:6827/2849203277","heartbeat_back_addr":"172.21.15.138:6831/2849203277","heartbeat_front_addr":"172.21.15.138:6829/2849203277","state":["exists","up"]},{"osd":1,"uuid":"d8d875df-c840-46cc-b6c0-a847f9c93999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6817","nonce":174521731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6819","nonce":174521731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6823","nonce":174521731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6821","nonce":174521731}]},"public_addr":"172.21.15.138:6817/174521731","cluster_addr":"172.21.15.138:6819/174521731","heartbeat_back_addr":"172.21.15.138:6823/174521731","heartbeat_front_addr":"172.21.15.138:6821/174521731","state":["exists","up"]},{"osd":2,"uuid":"4bd94373-76ca-47a6-870b-16d96d7c9e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6801","nonce":1365838748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6803","nonce":1365838748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6809","nonce":1365838748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6805","nonce":1365838748}]},"public_addr":"172.21.15.138:6801/1365838748","cluster_addr":"172.21.15.138:6803/1365838748","heartbeat_back_addr":"172.21.15.138:6809/1365838748","heartbeat_front_addr":"172.21.15.138:6805/1365838748","state":["exists","up"]},{"osd":3,"uuid":"07267087-d4d2-4e88-ab9b-bb87f6dd061a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6808","nonce":1841873885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6811","nonce":1841873885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6815","nonce":1841873885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6813","nonce":1841873885}]},"public_addr":"172.21.15.138:6808/1841873885","cluster_addr":"172.21.15.138:6811/1841873885","heartbeat_back_addr":"172.21.15.138:6815/1841873885","heartbeat_front_addr":"172.21.15.138:6813/1841873885","state":["exists","up"]},{"osd":4,"uuid":"b47ef758-d783-4168-a9a5-11db3de4ca12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6827","nonce":3960812298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6829","nonce":3960812298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6833","nonce":3960812298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6831","nonce":3960812298}]},"public_addr":"172.21.15.152:6827/3960812298","cluster_addr":"172.21.15.152:6829/3960812298","heartbeat_back_addr":"172.21.15.152:6833/3960812298","heartbeat_front_addr":"172.21.15.152:6831/3960812298","state":["exists","up"]},{"osd":5,"uuid":"0ea5d43e-61d8-4823-815f-9e42c62fae97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6819","nonce":4095234314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6821","nonce":4095234314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6825","nonce":4095234314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6823","nonce":4095234314}]},"public_addr":"172.21.15.152:6819/4095234314","cluster_addr":"172.21.15.152:6821/4095234314","heartbeat_back_addr":"172.21.15.152:6825/4095234314","heartbeat_front_addr":"172.21.15.152:6823/4095234314","state":["exists","up"]},{"osd":6,"uuid":"59ffac36-e2e0-4693-ac82-ad5eccf6224d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6803","nonce":223848127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6805","nonce":223848127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6809","nonce":223848127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6807","nonce":223848127}]},"public_addr":"172.21.15.152:6803/223848127","cluster_addr":"172.21.15.152:6805/223848127","heartbeat_back_addr":"172.21.15.152:6809/223848127","heartbeat_front_addr":"172.21.15.152:6807/223848127","state":["exists","up"]},{"osd":7,"uuid":"4f892c7d-32a5-4960-bf88-4ed7c863cd0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6811","nonce":52509095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6813","nonce":52509095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6817","nonce":52509095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6815","nonce":52509095}]},"public_addr":"172.21.15.152:6811/52509095","cluster_addr":"172.21.15.152:6813/52509095","heartbeat_back_addr":"172.21.15.152:6817/52509095","heartbeat_front_addr":"172.21.15.152:6815/52509095","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.434460+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.374709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.613134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.298148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.242144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.198102+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.093385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.197180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-21T22:52:44.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.989+0000 7f993ea00640 1 -- 172.21.15.138:0/1171067332 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f991006b380 msgr2=0x7f991006d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:44.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.989+0000 7f993ea00640 1 --2- 172.21.15.138:0/1171067332 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f991006b380 0x7f991006d840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f99380c5620 tx=0x7f9920007990 comp rx=0 tx=0).stop 2024-01-21T22:52:44.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.989+0000 7f993ea00640 1 -- 172.21.15.138:0/1171067332 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f993806b1c0 msgr2=0x7f9938147210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:44.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.989+0000 7f993ea00640 1 --2- 172.21.15.138:0/1171067332 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f993806b1c0 0x7f9938147210 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9928036040 tx=0x7f9928004a40 comp rx=0 tx=0).stop 2024-01-21T22:52:44.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.990+0000 7f993ea00640 1 -- 172.21.15.138:0/1171067332 shutdown_connections 2024-01-21T22:52:44.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.990+0000 7f993ea00640 1 --2- 172.21.15.138:0/1171067332 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f991006b380 0x7f991006d840 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:44.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.990+0000 7f993ea00640 1 --2- 172.21.15.138:0/1171067332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9938153a50 0x7f993815c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:44.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.990+0000 7f993ea00640 1 --2- 172.21.15.138:0/1171067332 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f993806c0a0 0x7f993815bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:44.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.990+0000 7f993ea00640 1 --2- 172.21.15.138:0/1171067332 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f993806b1c0 0x7f9938147210 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:44.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.990+0000 7f993ea00640 1 -- 172.21.15.138:0/1171067332 >> 172.21.15.138:0/1171067332 conn(0x7f993806d240 msgr2=0x7f99380c5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:44.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.990+0000 7f993ea00640 1 -- 172.21.15.138:0/1171067332 shutdown_connections 2024-01-21T22:52:44.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:44.990+0000 7f993ea00640 1 -- 172.21.15.138:0/1171067332 wait complete. 2024-01-21T22:52:46.008 INFO:tasks.ceph:Scrubbing osd.0 2024-01-21T22:52:46.008 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-01-21T22:52:46.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.232+0000 7f8243200640 1 Processor -- start 2024-01-21T22:52:46.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.233+0000 7f8243200640 1 -- start start 2024-01-21T22:52:46.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.234+0000 7f8243200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8244153ae0 0x7f824415a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:46.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.234+0000 7f8243200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82440c5200 0x7f82440c5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:46.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.234+0000 7f8243200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82440cb5e0 con 0x7f82440cf880 2024-01-21T22:52:46.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.234+0000 7f8243200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82440c4e10 con 0x7f8244153ae0 2024-01-21T22:52:46.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.234+0000 7f8243200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82440c5bb0 con 0x7f82440c5200 2024-01-21T22:52:46.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.235+0000 7f8241e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8244153ae0 0x7f824415a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:46.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.235+0000 7f8242800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82440c5200 0x7f82440c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:46.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.235+0000 7f8241e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8244153ae0 0x7f824415a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:48232/0 (socket says 172.21.15.138:48232) 2024-01-21T22:52:46.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.235+0000 7f8241e00640 1 -- 172.21.15.138:0/517086673 learned_addr learned my addr 172.21.15.138:0/517086673 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:46.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.239+0000 7f8242800640 1 -- 172.21.15.138:0/517086673 >> v2:172.21.15.152:3300/0 conn(0x7f8244153ae0 msgr2=0x7f824415a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:46.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.239+0000 7f8242800640 1 --2- 172.21.15.138:0/517086673 >> v2:172.21.15.152:3300/0 conn(0x7f8244153ae0 0x7f824415a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:46.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.239+0000 7f8242800640 1 -- 172.21.15.138:0/517086673 >> v1:172.21.15.152:6789/0 conn(0x7f82440cf880 legacy=0x7f82440c40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:46.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.239+0000 7f8242800640 1 -- 172.21.15.138:0/517086673 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82440cb830 con 0x7f82440c5200 2024-01-21T22:52:46.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.242+0000 7f8242800640 1 --2- 172.21.15.138:0/517086673 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82440c5200 0x7f82440c5670 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f82340097a0 tx=0x7f823402b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=332bef52c62b5ec8 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:46.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.247+0000 7f8240a00640 1 -- 172.21.15.138:0/517086673 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234030800 con 0x7f82440c5200 2024-01-21T22:52:46.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.247+0000 7f8240a00640 1 -- 172.21.15.138:0/517086673 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8234030960 con 0x7f82440c5200 2024-01-21T22:52:46.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.247+0000 7f8240a00640 1 -- 172.21.15.138:0/517086673 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234030cf0 con 0x7f82440c5200 2024-01-21T22:52:46.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.247+0000 7f8243200640 1 -- 172.21.15.138:0/517086673 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82440c5200 msgr2=0x7f82440c5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:46.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.247+0000 7f8243200640 1 --2- 172.21.15.138:0/517086673 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82440c5200 0x7f82440c5670 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f82340097a0 tx=0x7f823402b940 comp rx=0 tx=0).stop 2024-01-21T22:52:46.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.247+0000 7f8243200640 1 -- 172.21.15.138:0/517086673 shutdown_connections 2024-01-21T22:52:46.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.247+0000 7f8243200640 1 --2- 172.21.15.138:0/517086673 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f82440c5200 0x7f82440c5670 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:46.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.247+0000 7f8243200640 1 --2- 172.21.15.138:0/517086673 >> v2:172.21.15.152:3300/0 conn(0x7f8244153ae0 0x7f824415a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:46.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.247+0000 7f8243200640 1 -- 172.21.15.138:0/517086673 >> 172.21.15.138:0/517086673 conn(0x7f824406cd30 msgr2=0x7f824406d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:46.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.248+0000 7f8243200640 1 -- 172.21.15.138:0/517086673 shutdown_connections 2024-01-21T22:52:46.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.248+0000 7f8243200640 1 -- 172.21.15.138:0/517086673 wait complete. 2024-01-21T22:52:46.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.250+0000 7f8243200640 1 Processor -- start 2024-01-21T22:52:46.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.250+0000 7f8243200640 1 -- start start 2024-01-21T22:52:46.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.251+0000 7f8243200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82440c5200 0x7f824414ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:46.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.251+0000 7f8243200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82440cf880 0x7f82441e3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:46.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.251+0000 7f8243200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8244153ae0 0x7f82441e3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:46.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.251+0000 7f8243200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82441e5a50 con 0x7f82440cf880 2024-01-21T22:52:46.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.251+0000 7f8243200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82441e5490 con 0x7f82440c5200 2024-01-21T22:52:46.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.251+0000 7f8243200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82441e5770 con 0x7f8244153ae0 2024-01-21T22:52:46.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.251+0000 7f8241400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82440cf880 0x7f82441e3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:46.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.251+0000 7f8242800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8244153ae0 0x7f82441e3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:46.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.252+0000 7f8241400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82440cf880 0x7f82441e3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50932/0 (socket says 172.21.15.138:50932) 2024-01-21T22:52:46.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.252+0000 7f8241400640 1 -- 172.21.15.138:0/1175723788 learned_addr learned my addr 172.21.15.138:0/1175723788 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:46.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.255+0000 7f8241400640 1 -- 172.21.15.138:0/1175723788 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8244153ae0 msgr2=0x7f82441e3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:46.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.255+0000 7f8241400640 1 --2- 172.21.15.138:0/1175723788 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8244153ae0 0x7f82441e3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:46.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.255+0000 7f8241400640 1 -- 172.21.15.138:0/1175723788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82440c5200 msgr2=0x7f824414ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:46.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.255+0000 7f8241400640 1 --2- 172.21.15.138:0/1175723788 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f82440c5200 0x7f824414ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:46.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.255+0000 7f8241400640 1 -- 172.21.15.138:0/1175723788 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8234009400 con 0x7f82440cf880 2024-01-21T22:52:46.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.260+0000 7f8241400640 1 --2- 172.21.15.138:0/1175723788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82440cf880 0x7f82441e3010 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f822c00c730 tx=0x7f822c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:46.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.265+0000 7f823a000640 1 -- 172.21.15.138:0/1175723788 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c003b70 con 0x7f82440cf880 2024-01-21T22:52:46.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.265+0000 7f823a000640 1 -- 172.21.15.138:0/1175723788 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822c003cf0 con 0x7f82440cf880 2024-01-21T22:52:46.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.265+0000 7f823a000640 1 -- 172.21.15.138:0/1175723788 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822c003ea0 con 0x7f82440cf880 2024-01-21T22:52:46.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.266+0000 7f8243200640 1 -- 172.21.15.138:0/1175723788 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82440cea60 con 0x7f82440cf880 2024-01-21T22:52:46.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.266+0000 7f8243200640 1 -- 172.21.15.138:0/1175723788 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82440cd0b0 con 0x7f82440cf880 2024-01-21T22:52:46.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.267+0000 7f8243200640 1 -- 172.21.15.138:0/1175723788 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f82440cc0d0 con 0x7f82440cf880 2024-01-21T22:52:46.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.291+0000 7f823a000640 1 -- 172.21.15.138:0/1175723788 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f822c00aa90 con 0x7f82440cf880 2024-01-21T22:52:46.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.293+0000 7f823a000640 1 --2- 172.21.15.138:0/1175723788 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f821c06b380 0x7f821c06d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:46.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.293+0000 7f823a000640 1 -- 172.21.15.138:0/1175723788 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f822c00ce40 con 0x7f82440cf880 2024-01-21T22:52:46.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.293+0000 7f8241e00640 1 --2- 172.21.15.138:0/1175723788 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f821c06b380 0x7f821c06d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:46.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.293+0000 7f823a000640 1 --2- 172.21.15.138:0/1175723788 >> [v2:172.21.15.138:6824/2849203277,v1:172.21.15.138:6825/2849203277] conn(0x7f821c073580 0x7f821c0759a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:46.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.293+0000 7f823a000640 1 -- 172.21.15.138:0/1175723788 --> [v2:172.21.15.138:6824/2849203277,v1:172.21.15.138:6825/2849203277] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f821c076070 con 0x7f821c073580 2024-01-21T22:52:46.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.294+0000 7f8242800640 1 --2- 172.21.15.138:0/1175723788 >> [v2:172.21.15.138:6824/2849203277,v1:172.21.15.138:6825/2849203277] conn(0x7f821c073580 0x7f821c0759a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:46.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.294+0000 7f8241e00640 1 --2- 172.21.15.138:0/1175723788 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f821c06b380 0x7f821c06d840 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f823c0094a0 tx=0x7f823c0093f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:46.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.295+0000 7f8242800640 1 --2- 172.21.15.138:0/1175723788 >> [v2:172.21.15.138:6824/2849203277,v1:172.21.15.138:6825/2849203277] conn(0x7f821c073580 0x7f821c0759a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:46.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.296+0000 7f823a000640 1 -- 172.21.15.138:0/1175723788 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f822c058a40 con 0x7f82440cf880 2024-01-21T22:52:46.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.296+0000 7f823a000640 1 -- 172.21.15.138:0/1175723788 <== osd.0 v2:172.21.15.138:6824/2849203277 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f821c076070 con 0x7f821c073580 2024-01-21T22:52:46.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.314+0000 7f8243200640 1 -- 172.21.15.138:0/1175723788 --> [v2:172.21.15.138:6824/2849203277,v1:172.21.15.138:6825/2849203277] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f82440b52d0 con 0x7f821c073580 2024-01-21T22:52:46.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.314+0000 7f823a000640 1 -- 172.21.15.138:0/1175723788 <== osd.0 v2:172.21.15.138:6824/2849203277 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f82440b52d0 con 0x7f821c073580 2024-01-21T22:52:46.316 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-01-21T22:52:46.317 INFO:teuthology.orchestra.run.smithi138.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-21T22:52:46.317 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-01-21T22:52:46.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.315+0000 7f8243200640 1 -- 172.21.15.138:0/1175723788 >> [v2:172.21.15.138:6824/2849203277,v1:172.21.15.138:6825/2849203277] conn(0x7f821c073580 msgr2=0x7f821c0759a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:46.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.315+0000 7f8243200640 1 --2- 172.21.15.138:0/1175723788 >> [v2:172.21.15.138:6824/2849203277,v1:172.21.15.138:6825/2849203277] conn(0x7f821c073580 0x7f821c0759a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:46.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.315+0000 7f8243200640 1 -- 172.21.15.138:0/1175723788 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f821c06b380 msgr2=0x7f821c06d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:46.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.315+0000 7f8243200640 1 --2- 172.21.15.138:0/1175723788 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f821c06b380 0x7f821c06d840 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f823c0094a0 tx=0x7f823c0093f0 comp rx=0 tx=0).stop 2024-01-21T22:52:46.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.315+0000 7f8243200640 1 -- 172.21.15.138:0/1175723788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82440cf880 msgr2=0x7f82441e3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:46.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.315+0000 7f8243200640 1 --2- 172.21.15.138:0/1175723788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82440cf880 0x7f82441e3010 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f822c00c730 tx=0x7f822c00cbf0 comp rx=0 tx=0).stop 2024-01-21T22:52:46.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.315+0000 7f8242800640 1 -- 172.21.15.138:0/1175723788 reap_dead start 2024-01-21T22:52:46.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.315+0000 7f8243200640 1 -- 172.21.15.138:0/1175723788 shutdown_connections 2024-01-21T22:52:46.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.315+0000 7f8243200640 1 -- 172.21.15.138:0/1175723788 >> 172.21.15.138:0/1175723788 conn(0x7f824406cd30 msgr2=0x7f82440ca8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:46.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.316+0000 7f8243200640 1 -- 172.21.15.138:0/1175723788 shutdown_connections 2024-01-21T22:52:46.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.316+0000 7f8243200640 1 -- 172.21.15.138:0/1175723788 wait complete. 2024-01-21T22:52:46.339 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-01-21T22:52:46.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.574+0000 7f7aa6000640 1 Processor -- start 2024-01-21T22:52:46.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.574+0000 7f7aa6000640 1 -- start start 2024-01-21T22:52:46.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.576+0000 7f7aa6000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7aa0151c60 0x7f7aa0157720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:46.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.576+0000 7f7aa6000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7aa00c26e0 0x7f7aa01620a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:46.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.576+0000 7f7aa6000640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa00c3fd0 con 0x7f7aa006e690 2024-01-21T22:52:46.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.576+0000 7f7aa6000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aa006e4e0 con 0x7f7aa00c26e0 2024-01-21T22:52:46.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.576+0000 7f7aa6000640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa00c7f80 con 0x7f7aa0151c60 2024-01-21T22:52:46.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.601+0000 7f7a9f400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7aa0151c60 0x7f7aa0157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:46.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.602+0000 7f7a9f400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7aa0151c60 0x7f7aa0157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:55258/0 (socket says 172.21.15.138:55258) 2024-01-21T22:52:46.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.602+0000 7f7a9f400640 1 -- 172.21.15.138:0/29875408 learned_addr learned my addr 172.21.15.138:0/29875408 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:46.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.605+0000 7f7a9fe00640 1 --2- 172.21.15.138:0/29875408 >> v2:172.21.15.152:3300/0 conn(0x7f7aa00c26e0 0x7f7aa01620a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:46.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.608+0000 7f7a9f400640 1 -- 172.21.15.138:0/29875408 >> v2:172.21.15.152:3300/0 conn(0x7f7aa00c26e0 msgr2=0x7f7aa01620a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:46.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.608+0000 7f7a9f400640 1 --2- 172.21.15.138:0/29875408 >> v2:172.21.15.152:3300/0 conn(0x7f7aa00c26e0 0x7f7aa01620a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:46.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.608+0000 7f7a9f400640 1 -- 172.21.15.138:0/29875408 >> v1:172.21.15.152:6789/0 conn(0x7f7aa006e690 legacy=0x7f7aa00cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:46.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.608+0000 7f7a9f400640 1 -- 172.21.15.138:0/29875408 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa00c7ca0 con 0x7f7aa0151c60 2024-01-21T22:52:46.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.611+0000 7f7a9f400640 1 --2- 172.21.15.138:0/29875408 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7aa0151c60 0x7f7aa0157720 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009870 tx=0x7f7a9002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2b5a3bf5b4fbc7b server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:46.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.613+0000 7f7a9e000640 1 -- 172.21.15.138:0/29875408 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a90030840 con 0x7f7aa0151c60 2024-01-21T22:52:46.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.615+0000 7f7a9e000640 1 -- 172.21.15.138:0/29875408 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a900309a0 con 0x7f7aa0151c60 2024-01-21T22:52:46.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.615+0000 7f7a9e000640 1 -- 172.21.15.138:0/29875408 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a90030d30 con 0x7f7aa0151c60 2024-01-21T22:52:46.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.615+0000 7f7aa6000640 1 -- 172.21.15.138:0/29875408 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7aa0151c60 msgr2=0x7f7aa0157720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:46.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.615+0000 7f7aa6000640 1 --2- 172.21.15.138:0/29875408 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7aa0151c60 0x7f7aa0157720 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009870 tx=0x7f7a9002b980 comp rx=0 tx=0).stop 2024-01-21T22:52:46.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.615+0000 7f7aa6000640 1 -- 172.21.15.138:0/29875408 shutdown_connections 2024-01-21T22:52:46.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.616+0000 7f7aa6000640 1 --2- 172.21.15.138:0/29875408 >> v2:172.21.15.152:3300/0 conn(0x7f7aa00c26e0 0x7f7aa01620a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:46.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.616+0000 7f7aa6000640 1 --2- 172.21.15.138:0/29875408 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7aa0151c60 0x7f7aa0157720 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:46.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.616+0000 7f7aa6000640 1 -- 172.21.15.138:0/29875408 >> 172.21.15.138:0/29875408 conn(0x7f7aa0072810 msgr2=0x7f7aa0066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:46.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.616+0000 7f7aa6000640 1 -- 172.21.15.138:0/29875408 shutdown_connections 2024-01-21T22:52:46.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.616+0000 7f7aa6000640 1 -- 172.21.15.138:0/29875408 wait complete. 2024-01-21T22:52:46.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.618+0000 7f7aa6000640 1 Processor -- start 2024-01-21T22:52:46.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.618+0000 7f7aa6000640 1 -- start start 2024-01-21T22:52:46.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.619+0000 7f7aa6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7aa006e690 0x7f7aa01ee500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:46.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.619+0000 7f7aa6000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7aa00c26e0 0x7f7aa01569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:46.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.619+0000 7f7aa6000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7aa015b520 0x7f7aa014a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:46.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.619+0000 7f7aa6000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa01ee070 con 0x7f7aa006e690 2024-01-21T22:52:46.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.619+0000 7f7aa6000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa01f1d40 con 0x7f7aa00c26e0 2024-01-21T22:52:46.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.619+0000 7f7aa6000640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa01f1780 con 0x7f7aa015b520 2024-01-21T22:52:46.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.620+0000 7f7a9fe00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7aa015b520 0x7f7aa014a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:46.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.620+0000 7f7a9ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7aa00c26e0 0x7f7aa01569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:46.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.620+0000 7f7a9ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7aa00c26e0 0x7f7aa01569e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:48264/0 (socket says 172.21.15.138:48264) 2024-01-21T22:52:46.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.620+0000 7f7a9ea00640 1 -- 172.21.15.138:0/1619022597 learned_addr learned my addr 172.21.15.138:0/1619022597 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:46.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.623+0000 7f7a9ea00640 1 -- 172.21.15.138:0/1619022597 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7aa015b520 msgr2=0x7f7aa014a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:46.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.623+0000 7f7a9ea00640 1 --2- 172.21.15.138:0/1619022597 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7aa015b520 0x7f7aa014a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:46.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.623+0000 7f7a9ea00640 1 -- 172.21.15.138:0/1619022597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7aa006e690 msgr2=0x7f7aa01ee500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:46.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.623+0000 7f7a9ea00640 1 --2- 172.21.15.138:0/1619022597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7aa006e690 0x7f7aa01ee500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:46.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.623+0000 7f7a9ea00640 1 -- 172.21.15.138:0/1619022597 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a900094d0 con 0x7f7aa00c26e0 2024-01-21T22:52:46.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.627+0000 7f7a9ea00640 1 --2- 172.21.15.138:0/1619022597 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7aa00c26e0 0x7f7aa01569e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800c430 tx=0x7f7a8800c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:46.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.680+0000 7f7a97400640 1 -- 172.21.15.138:0/1619022597 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88011790 con 0x7f7aa00c26e0 2024-01-21T22:52:46.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.680+0000 7f7a97400640 1 -- 172.21.15.138:0/1619022597 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a880118f0 con 0x7f7aa00c26e0 2024-01-21T22:52:46.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.680+0000 7f7a97400640 1 -- 172.21.15.138:0/1619022597 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88011ce0 con 0x7f7aa00c26e0 2024-01-21T22:52:46.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.680+0000 7f7aa6000640 1 -- 172.21.15.138:0/1619022597 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa01f1a60 con 0x7f7aa00c26e0 2024-01-21T22:52:46.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.681+0000 7f7aa6000640 1 -- 172.21.15.138:0/1619022597 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa01edd00 con 0x7f7aa00c26e0 2024-01-21T22:52:46.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.682+0000 7f7aa6000640 1 -- 172.21.15.138:0/1619022597 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa004ea30 con 0x7f7aa00c26e0 2024-01-21T22:52:46.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.933+0000 7f7a97400640 1 -- 172.21.15.138:0/1619022597 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f7a8801c460 con 0x7f7aa00c26e0 2024-01-21T22:52:46.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.935+0000 7f7a97400640 1 --2- 172.21.15.138:0/1619022597 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7a7806b380 0x7f7a7806d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:46.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.935+0000 7f7a97400640 1 -- 172.21.15.138:0/1619022597 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7a8808a550 con 0x7f7aa00c26e0 2024-01-21T22:52:46.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.935+0000 7f7a9f400640 1 --2- 172.21.15.138:0/1619022597 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7a7806b380 0x7f7a7806d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:46.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.936+0000 7f7a9f400640 1 --2- 172.21.15.138:0/1619022597 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7a7806b380 0x7f7a7806d840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009870 tx=0x7f7a9003b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:46.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:46.945+0000 7f7a97400640 1 -- 172.21.15.138:0/1619022597 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f7a88055210 con 0x7f7aa00c26e0 2024-01-21T22:52:47.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.150+0000 7f7aa6000640 1 -- 172.21.15.138:0/1619022597 --> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f7aa00c21c0 con 0x7f7a7806b380 2024-01-21T22:52:47.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.151+0000 7f7a97400640 1 -- 172.21.15.138:0/1619022597 <== mgr.4111 v2:172.21.15.152:6800/314625564 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7a90030b10 con 0x7f7a7806b380 2024-01-21T22:52:47.152 INFO:teuthology.orchestra.run.smithi138.stderr:instructed osd(s) 0 to deep-scrub 2024-01-21T22:52:47.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.153+0000 7f7aa6000640 1 -- 172.21.15.138:0/1619022597 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7a7806b380 msgr2=0x7f7a7806d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:47.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.153+0000 7f7aa6000640 1 --2- 172.21.15.138:0/1619022597 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7a7806b380 0x7f7a7806d840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009870 tx=0x7f7a9003b040 comp rx=0 tx=0).stop 2024-01-21T22:52:47.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.153+0000 7f7aa6000640 1 -- 172.21.15.138:0/1619022597 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7aa00c26e0 msgr2=0x7f7aa01569e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:47.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.153+0000 7f7aa6000640 1 --2- 172.21.15.138:0/1619022597 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7aa00c26e0 0x7f7aa01569e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800c430 tx=0x7f7a8800c8f0 comp rx=0 tx=0).stop 2024-01-21T22:52:47.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.153+0000 7f7aa6000640 1 -- 172.21.15.138:0/1619022597 shutdown_connections 2024-01-21T22:52:47.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.153+0000 7f7aa6000640 1 --2- 172.21.15.138:0/1619022597 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7a7806b380 0x7f7a7806d840 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:47.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.153+0000 7f7aa6000640 1 --2- 172.21.15.138:0/1619022597 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7aa015b520 0x7f7aa014a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:47.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.153+0000 7f7aa6000640 1 --2- 172.21.15.138:0/1619022597 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7aa00c26e0 0x7f7aa01569e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:47.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.153+0000 7f7aa6000640 1 --2- 172.21.15.138:0/1619022597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7aa006e690 0x7f7aa01ee500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:47.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.153+0000 7f7aa6000640 1 -- 172.21.15.138:0/1619022597 >> 172.21.15.138:0/1619022597 conn(0x7f7aa0072810 msgr2=0x7f7aa00c9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:47.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.154+0000 7f7aa6000640 1 -- 172.21.15.138:0/1619022597 shutdown_connections 2024-01-21T22:52:47.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.154+0000 7f7aa6000640 1 -- 172.21.15.138:0/1619022597 wait complete. 2024-01-21T22:52:47.174 INFO:tasks.ceph:Scrubbing osd.1 2024-01-21T22:52:47.174 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-01-21T22:52:47.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.383+0000 7f0672200640 1 Processor -- start 2024-01-21T22:52:47.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.383+0000 7f0672200640 1 -- start start 2024-01-21T22:52:47.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.385+0000 7f0672200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f066c153d30 0x7f066c15a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:47.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.385+0000 7f0672200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f066c0c5130 0x7f066c164300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:47.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.385+0000 7f0672200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f066c154110 con 0x7f066c0cad90 2024-01-21T22:52:47.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.385+0000 7f0672200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f066c0c4730 con 0x7f066c0c5130 2024-01-21T22:52:47.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.385+0000 7f0672200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066c06a820 con 0x7f066c153d30 2024-01-21T22:52:47.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.385+0000 7f066b400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f066c153d30 0x7f066c15a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:47.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.387+0000 7f066b400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f066c153d30 0x7f066c15a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:55270/0 (socket says 172.21.15.138:55270) 2024-01-21T22:52:47.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.387+0000 7f066b400640 1 -- 172.21.15.138:0/876961696 learned_addr learned my addr 172.21.15.138:0/876961696 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:47.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.387+0000 7f066be00640 1 --2- 172.21.15.138:0/876961696 >> v2:172.21.15.138:3300/0 conn(0x7f066c0c5130 0x7f066c164300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.390+0000 7f066b400640 1 -- 172.21.15.138:0/876961696 >> v2:172.21.15.138:3300/0 conn(0x7f066c0c5130 msgr2=0x7f066c164300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.390+0000 7f066b400640 1 --2- 172.21.15.138:0/876961696 >> v2:172.21.15.138:3300/0 conn(0x7f066c0c5130 0x7f066c164300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.390+0000 7f066b400640 1 -- 172.21.15.138:0/876961696 >> v1:172.21.15.138:6789/0 conn(0x7f066c0cad90 legacy=0x7f066c0c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.390+0000 7f066b400640 1 -- 172.21.15.138:0/876961696 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f066c15a940 con 0x7f066c153d30 2024-01-21T22:52:47.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.392+0000 7f066b400640 1 --2- 172.21.15.138:0/876961696 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f066c153d30 0x7f066c15a400 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f065c0098a0 tx=0x7f065c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5932aa2ab5f3818 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:47.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.396+0000 7f066a000640 1 -- 172.21.15.138:0/876961696 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c030840 con 0x7f066c153d30 2024-01-21T22:52:47.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.396+0000 7f066a000640 1 -- 172.21.15.138:0/876961696 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f065c0309a0 con 0x7f066c153d30 2024-01-21T22:52:47.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.396+0000 7f066a000640 1 -- 172.21.15.138:0/876961696 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065c030d30 con 0x7f066c153d30 2024-01-21T22:52:47.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.396+0000 7f0672200640 1 -- 172.21.15.138:0/876961696 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f066c153d30 msgr2=0x7f066c15a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:47.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.396+0000 7f0672200640 1 --2- 172.21.15.138:0/876961696 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f066c153d30 0x7f066c15a400 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f065c0098a0 tx=0x7f065c02b980 comp rx=0 tx=0).stop 2024-01-21T22:52:47.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.397+0000 7f0672200640 1 -- 172.21.15.138:0/876961696 shutdown_connections 2024-01-21T22:52:47.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.397+0000 7f0672200640 1 --2- 172.21.15.138:0/876961696 >> v2:172.21.15.138:3300/0 conn(0x7f066c0c5130 0x7f066c164300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:47.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.397+0000 7f0672200640 1 --2- 172.21.15.138:0/876961696 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f066c153d30 0x7f066c15a400 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:47.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.397+0000 7f0672200640 1 -- 172.21.15.138:0/876961696 >> 172.21.15.138:0/876961696 conn(0x7f066c06cee0 msgr2=0x7f066c066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:47.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.397+0000 7f0672200640 1 -- 172.21.15.138:0/876961696 shutdown_connections 2024-01-21T22:52:47.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.397+0000 7f0672200640 1 -- 172.21.15.138:0/876961696 wait complete. 2024-01-21T22:52:47.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.398+0000 7f0672200640 1 Processor -- start 2024-01-21T22:52:47.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.399+0000 7f0672200640 1 -- start start 2024-01-21T22:52:47.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.399+0000 7f0672200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f066c0cad90 0x7f066c1e4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:47.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.399+0000 7f0672200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f066c153d30 0x7f066c0cf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:47.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.400+0000 7f0672200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f066c145d30 0x7f066c1461c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:47.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.400+0000 7f0672200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066c14c7d0 con 0x7f066c145d30 2024-01-21T22:52:47.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.400+0000 7f0672200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f066c1e55a0 con 0x7f066c0cad90 2024-01-21T22:52:47.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.400+0000 7f0672200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066c1e2d90 con 0x7f066c153d30 2024-01-21T22:52:47.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.400+0000 7f066aa00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f066c153d30 0x7f066c0cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:47.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.400+0000 7f066be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f066c145d30 0x7f066c1461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:47.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.400+0000 7f066b400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f066c0cad90 0x7f066c1e4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:47.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.401+0000 7f066aa00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f066c153d30 0x7f066c0cf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:55274/0 (socket says 172.21.15.138:55274) 2024-01-21T22:52:47.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.401+0000 7f066aa00640 1 -- 172.21.15.138:0/2369055558 learned_addr learned my addr 172.21.15.138:0/2369055558 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:47.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.404+0000 7f066aa00640 1 -- 172.21.15.138:0/2369055558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f066c0cad90 msgr2=0x7f066c1e4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:47.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.404+0000 7f066aa00640 1 --2- 172.21.15.138:0/2369055558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f066c0cad90 0x7f066c1e4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:47.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.404+0000 7f066aa00640 1 -- 172.21.15.138:0/2369055558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f066c145d30 msgr2=0x7f066c1461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:47.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.404+0000 7f066aa00640 1 --2- 172.21.15.138:0/2369055558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f066c145d30 0x7f066c1461c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:47.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.404+0000 7f066aa00640 1 -- 172.21.15.138:0/2369055558 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f065c0094d0 con 0x7f066c153d30 2024-01-21T22:52:47.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.407+0000 7f066aa00640 1 --2- 172.21.15.138:0/2369055558 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f066c153d30 0x7f066c0cf180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f065400c400 tx=0x7f065400c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:47.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.411+0000 7f0663400640 1 -- 172.21.15.138:0/2369055558 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06540052d0 con 0x7f066c153d30 2024-01-21T22:52:47.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.411+0000 7f0663400640 1 -- 172.21.15.138:0/2369055558 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0654005450 con 0x7f066c153d30 2024-01-21T22:52:47.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.411+0000 7f0663400640 1 -- 172.21.15.138:0/2369055558 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654007260 con 0x7f066c153d30 2024-01-21T22:52:47.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.411+0000 7f0672200640 1 -- 172.21.15.138:0/2369055558 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f066c1e5300 con 0x7f066c153d30 2024-01-21T22:52:47.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.411+0000 7f0672200640 1 -- 172.21.15.138:0/2369055558 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f066c0cd760 con 0x7f066c153d30 2024-01-21T22:52:47.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.412+0000 7f0672200640 1 -- 172.21.15.138:0/2369055558 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f066c06ada0 con 0x7f066c153d30 2024-01-21T22:52:47.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.433+0000 7f0663400640 1 -- 172.21.15.138:0/2369055558 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f06540073c0 con 0x7f066c153d30 2024-01-21T22:52:47.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.434+0000 7f0663400640 1 --2- 172.21.15.138:0/2369055558 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f064406b380 0x7f064406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:47.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.434+0000 7f0663400640 1 -- 172.21.15.138:0/2369055558 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f0654089680 con 0x7f066c153d30 2024-01-21T22:52:47.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.434+0000 7f0663400640 1 --2- 172.21.15.138:0/2369055558 >> [v2:172.21.15.138:6816/174521731,v1:172.21.15.138:6817/174521731] conn(0x7f0644073580 0x7f06440759a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:47.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.434+0000 7f0663400640 1 -- 172.21.15.138:0/2369055558 --> [v2:172.21.15.138:6816/174521731,v1:172.21.15.138:6817/174521731] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0644076070 con 0x7f0644073580 2024-01-21T22:52:47.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.434+0000 7f066b400640 1 --2- 172.21.15.138:0/2369055558 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f064406b380 0x7f064406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:47.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.435+0000 7f066be00640 1 --2- 172.21.15.138:0/2369055558 >> [v2:172.21.15.138:6816/174521731,v1:172.21.15.138:6817/174521731] conn(0x7f0644073580 0x7f06440759a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:47.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.435+0000 7f0663400640 1 -- 172.21.15.138:0/2369055558 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0654054480 con 0x7f066c153d30 2024-01-21T22:52:47.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.435+0000 7f066b400640 1 --2- 172.21.15.138:0/2369055558 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f064406b380 0x7f064406d840 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f065c036000 tx=0x7f065c008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:47.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.435+0000 7f066be00640 1 --2- 172.21.15.138:0/2369055558 >> [v2:172.21.15.138:6816/174521731,v1:172.21.15.138:6817/174521731] conn(0x7f0644073580 0x7f06440759a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:47.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.437+0000 7f0663400640 1 -- 172.21.15.138:0/2369055558 <== osd.1 v2:172.21.15.138:6816/174521731 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0644076070 con 0x7f0644073580 2024-01-21T22:52:47.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.454+0000 7f0672200640 1 -- 172.21.15.138:0/2369055558 --> [v2:172.21.15.138:6816/174521731,v1:172.21.15.138:6817/174521731] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f066c06aac0 con 0x7f0644073580 2024-01-21T22:52:47.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.455+0000 7f0663400640 1 -- 172.21.15.138:0/2369055558 <== osd.1 v2:172.21.15.138:6816/174521731 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f066400f040 con 0x7f0644073580 2024-01-21T22:52:47.456 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-01-21T22:52:47.457 INFO:teuthology.orchestra.run.smithi138.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-21T22:52:47.457 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-01-21T22:52:47.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.455+0000 7f0672200640 1 -- 172.21.15.138:0/2369055558 >> [v2:172.21.15.138:6816/174521731,v1:172.21.15.138:6817/174521731] conn(0x7f0644073580 msgr2=0x7f06440759a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:47.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.456+0000 7f0672200640 1 --2- 172.21.15.138:0/2369055558 >> [v2:172.21.15.138:6816/174521731,v1:172.21.15.138:6817/174521731] conn(0x7f0644073580 0x7f06440759a0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:47.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.456+0000 7f0672200640 1 -- 172.21.15.138:0/2369055558 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f064406b380 msgr2=0x7f064406d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:47.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.456+0000 7f0672200640 1 --2- 172.21.15.138:0/2369055558 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f064406b380 0x7f064406d840 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f065c036000 tx=0x7f065c008030 comp rx=0 tx=0).stop 2024-01-21T22:52:47.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.456+0000 7f0672200640 1 -- 172.21.15.138:0/2369055558 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f066c153d30 msgr2=0x7f066c0cf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:47.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.456+0000 7f0672200640 1 --2- 172.21.15.138:0/2369055558 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f066c153d30 0x7f066c0cf180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f065400c400 tx=0x7f065400c8c0 comp rx=0 tx=0).stop 2024-01-21T22:52:47.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.456+0000 7f066be00640 1 -- 172.21.15.138:0/2369055558 reap_dead start 2024-01-21T22:52:47.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.456+0000 7f0672200640 1 -- 172.21.15.138:0/2369055558 shutdown_connections 2024-01-21T22:52:47.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.456+0000 7f0672200640 1 -- 172.21.15.138:0/2369055558 >> 172.21.15.138:0/2369055558 conn(0x7f066c06cee0 msgr2=0x7f066c0cb1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:47.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.456+0000 7f0672200640 1 -- 172.21.15.138:0/2369055558 shutdown_connections 2024-01-21T22:52:47.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.456+0000 7f0672200640 1 -- 172.21.15.138:0/2369055558 wait complete. 2024-01-21T22:52:47.476 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-01-21T22:52:47.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.689+0000 7f01e9400640 1 Processor -- start 2024-01-21T22:52:47.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.690+0000 7f01e9400640 1 -- start start 2024-01-21T22:52:47.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.691+0000 7f01e9400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f01e406c5d0 0x7f01e40c6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:47.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.691+0000 7f01e9400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01e40c5970 0x7f01e40c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:47.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.691+0000 7f01e9400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e40c55a0 con 0x7f01e406c9b0 2024-01-21T22:52:47.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.691+0000 7f01e9400640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e40c3d30 con 0x7f01e406c5d0 2024-01-21T22:52:47.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.691+0000 7f01e9400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e40c4310 con 0x7f01e40c5970 2024-01-21T22:52:47.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.710+0000 7f01e3200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01e40c5970 0x7f01e40c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:47.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.710+0000 7f01e3200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01e40c5970 0x7f01e40c5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:55280/0 (socket says 172.21.15.138:55280) 2024-01-21T22:52:47.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.710+0000 7f01e3200640 1 -- 172.21.15.138:0/396771897 learned_addr learned my addr 172.21.15.138:0/396771897 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:47.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.713+0000 7f01e3200640 1 -- 172.21.15.138:0/396771897 >> v2:172.21.15.138:3300/0 conn(0x7f01e406c5d0 msgr2=0x7f01e40c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:47.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.713+0000 7f01e3200640 1 --2- 172.21.15.138:0/396771897 >> v2:172.21.15.138:3300/0 conn(0x7f01e406c5d0 0x7f01e40c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:47.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.713+0000 7f01e3200640 1 -- 172.21.15.138:0/396771897 >> v1:172.21.15.138:6789/0 conn(0x7f01e406c9b0 legacy=0x7f01e406d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:47.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.713+0000 7f01e3200640 1 -- 172.21.15.138:0/396771897 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e40c28d0 con 0x7f01e40c5970 2024-01-21T22:52:47.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.716+0000 7f01e3200640 1 --2- 172.21.15.138:0/396771897 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01e40c5970 0x7f01e40c5060 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f01e4072600 tx=0x7f01dc02bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=316adaa076c409e2 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:47.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.720+0000 7f01e1e00640 1 -- 172.21.15.138:0/396771897 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc02cae0 con 0x7f01e40c5970 2024-01-21T22:52:47.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.720+0000 7f01e1e00640 1 -- 172.21.15.138:0/396771897 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01dc02cc40 con 0x7f01e40c5970 2024-01-21T22:52:47.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.720+0000 7f01e1e00640 1 -- 172.21.15.138:0/396771897 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc0343f0 con 0x7f01e40c5970 2024-01-21T22:52:47.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.721+0000 7f01e9400640 1 -- 172.21.15.138:0/396771897 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01e40c5970 msgr2=0x7f01e40c5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:47.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.721+0000 7f01e9400640 1 --2- 172.21.15.138:0/396771897 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01e40c5970 0x7f01e40c5060 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f01e4072600 tx=0x7f01dc02bf40 comp rx=0 tx=0).stop 2024-01-21T22:52:47.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.721+0000 7f01e9400640 1 -- 172.21.15.138:0/396771897 shutdown_connections 2024-01-21T22:52:47.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.721+0000 7f01e9400640 1 --2- 172.21.15.138:0/396771897 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01e40c5970 0x7f01e40c5060 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:47.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.721+0000 7f01e9400640 1 --2- 172.21.15.138:0/396771897 >> v2:172.21.15.138:3300/0 conn(0x7f01e406c5d0 0x7f01e40c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:47.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.721+0000 7f01e9400640 1 -- 172.21.15.138:0/396771897 >> 172.21.15.138:0/396771897 conn(0x7f01e40cb070 msgr2=0x7f01e40cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:47.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.721+0000 7f01e9400640 1 -- 172.21.15.138:0/396771897 shutdown_connections 2024-01-21T22:52:47.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.721+0000 7f01e9400640 1 -- 172.21.15.138:0/396771897 wait complete. 2024-01-21T22:52:47.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.724+0000 7f01e9400640 1 Processor -- start 2024-01-21T22:52:47.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.724+0000 7f01e9400640 1 -- start start 2024-01-21T22:52:47.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.725+0000 7f01e9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01e406c5d0 0x7f01e4154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:47.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.725+0000 7f01e9400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01e406c9b0 0x7f01e414a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:47.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.725+0000 7f01e9400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01e40c5970 0x7f01e406b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:47.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.726+0000 7f01e9400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e4157df0 con 0x7f01e406c5d0 2024-01-21T22:52:47.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.726+0000 7f01e9400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e4153ca0 con 0x7f01e406c9b0 2024-01-21T22:52:47.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.726+0000 7f01e9400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e4157830 con 0x7f01e40c5970 2024-01-21T22:52:47.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.726+0000 7f01e3200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01e40c5970 0x7f01e406b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:47.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.726+0000 7f01dbe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01e406c9b0 0x7f01e414a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:47.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.727+0000 7f01e3200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01e40c5970 0x7f01e406b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:55286/0 (socket says 172.21.15.138:55286) 2024-01-21T22:52:47.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.727+0000 7f01e3200640 1 -- 172.21.15.138:0/2775822179 learned_addr learned my addr 172.21.15.138:0/2775822179 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:47.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.730+0000 7f01e3200640 1 -- 172.21.15.138:0/2775822179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01e406c9b0 msgr2=0x7f01e414a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:47.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.730+0000 7f01e3200640 1 --2- 172.21.15.138:0/2775822179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01e406c9b0 0x7f01e414a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:47.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.730+0000 7f01e3200640 1 -- 172.21.15.138:0/2775822179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01e406c5d0 msgr2=0x7f01e4154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:47.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.730+0000 7f01e3200640 1 --2- 172.21.15.138:0/2775822179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01e406c5d0 0x7f01e4154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:47.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.730+0000 7f01e3200640 1 -- 172.21.15.138:0/2775822179 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01dc009b30 con 0x7f01e40c5970 2024-01-21T22:52:47.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.732+0000 7f01e3200640 1 --2- 172.21.15.138:0/2775822179 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01e40c5970 0x7f01e406b550 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f01dc036040 tx=0x7f01dc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:47.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.735+0000 7f01daa00640 1 -- 172.21.15.138:0/2775822179 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc03e450 con 0x7f01e40c5970 2024-01-21T22:52:47.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.735+0000 7f01daa00640 1 -- 172.21.15.138:0/2775822179 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01dc03e5d0 con 0x7f01e40c5970 2024-01-21T22:52:47.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.735+0000 7f01daa00640 1 -- 172.21.15.138:0/2775822179 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc03e980 con 0x7f01e40c5970 2024-01-21T22:52:47.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.735+0000 7f01e9400640 1 -- 172.21.15.138:0/2775822179 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e4157b10 con 0x7f01e40c5970 2024-01-21T22:52:47.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.736+0000 7f01e9400640 1 -- 172.21.15.138:0/2775822179 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e4154860 con 0x7f01e40c5970 2024-01-21T22:52:47.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.737+0000 7f01e9400640 1 -- 172.21.15.138:0/2775822179 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e41552c0 con 0x7f01e40c5970 2024-01-21T22:52:47.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.756+0000 7f01daa00640 1 -- 172.21.15.138:0/2775822179 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f01dc03eae0 con 0x7f01e40c5970 2024-01-21T22:52:47.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.758+0000 7f01daa00640 1 --2- 172.21.15.138:0/2775822179 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f01bc06b380 0x7f01bc06d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:47.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.758+0000 7f01e2800640 1 --2- 172.21.15.138:0/2775822179 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f01bc06b380 0x7f01bc06d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:47.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.758+0000 7f01daa00640 1 -- 172.21.15.138:0/2775822179 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f01dc07bed0 con 0x7f01e40c5970 2024-01-21T22:52:47.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.759+0000 7f01e2800640 1 --2- 172.21.15.138:0/2775822179 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f01bc06b380 0x7f01bc06d840 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009710 tx=0x7f01d4006c90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:47.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:47.995+0000 7f01daa00640 1 -- 172.21.15.138:0/2775822179 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f01dc077b10 con 0x7f01e40c5970 2024-01-21T22:52:48.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.199+0000 7f01e9400640 1 -- 172.21.15.138:0/2775822179 --> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f01e40c4580 con 0x7f01bc06b380 2024-01-21T22:52:48.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.200+0000 7f01daa00640 1 -- 172.21.15.138:0/2775822179 <== mgr.4111 v2:172.21.15.152:6800/314625564 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f01d400b480 con 0x7f01bc06b380 2024-01-21T22:52:48.202 INFO:teuthology.orchestra.run.smithi138.stderr:instructed osd(s) 1 to deep-scrub 2024-01-21T22:52:48.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.203+0000 7f01e9400640 1 -- 172.21.15.138:0/2775822179 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f01bc06b380 msgr2=0x7f01bc06d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:48.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.203+0000 7f01e9400640 1 --2- 172.21.15.138:0/2775822179 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f01bc06b380 0x7f01bc06d840 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f01d4009710 tx=0x7f01d4006c90 comp rx=0 tx=0).stop 2024-01-21T22:52:48.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.203+0000 7f01e9400640 1 -- 172.21.15.138:0/2775822179 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01e40c5970 msgr2=0x7f01e406b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:48.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.203+0000 7f01e9400640 1 --2- 172.21.15.138:0/2775822179 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01e40c5970 0x7f01e406b550 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f01dc036040 tx=0x7f01dc004a40 comp rx=0 tx=0).stop 2024-01-21T22:52:48.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.203+0000 7f01e9400640 1 -- 172.21.15.138:0/2775822179 shutdown_connections 2024-01-21T22:52:48.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.203+0000 7f01e9400640 1 --2- 172.21.15.138:0/2775822179 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f01bc06b380 0x7f01bc06d840 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:48.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.203+0000 7f01e9400640 1 --2- 172.21.15.138:0/2775822179 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f01e40c5970 0x7f01e406b550 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:48.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.203+0000 7f01e9400640 1 --2- 172.21.15.138:0/2775822179 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f01e406c9b0 0x7f01e414a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:48.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.203+0000 7f01e9400640 1 --2- 172.21.15.138:0/2775822179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01e406c5d0 0x7f01e4154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:48.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.203+0000 7f01e9400640 1 -- 172.21.15.138:0/2775822179 >> 172.21.15.138:0/2775822179 conn(0x7f01e40cb070 msgr2=0x7f01e40ca560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:48.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.203+0000 7f01e9400640 1 -- 172.21.15.138:0/2775822179 shutdown_connections 2024-01-21T22:52:48.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.203+0000 7f01e9400640 1 -- 172.21.15.138:0/2775822179 wait complete. 2024-01-21T22:52:48.223 INFO:tasks.ceph:Scrubbing osd.2 2024-01-21T22:52:48.223 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-01-21T22:52:48.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.438+0000 7f8e0f200640 1 Processor -- start 2024-01-21T22:52:48.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.439+0000 7f8e0f200640 1 -- start start 2024-01-21T22:52:48.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.440+0000 7f8e0f200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8e080cad90 0x7f8e080c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:48.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.440+0000 7f8e0f200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8e080cb1d0 0x7f8e080c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:48.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.440+0000 7f8e0f200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e08154110 con 0x7f8e08153d30 2024-01-21T22:52:48.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.440+0000 7f8e0f200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e080c4730 con 0x7f8e080cad90 2024-01-21T22:52:48.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.440+0000 7f8e0f200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e0806a820 con 0x7f8e080cb1d0 2024-01-21T22:52:48.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.441+0000 7f8e07400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8e080cad90 0x7f8e080c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:48.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.441+0000 7f8e0d200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f8e080cb1d0 0x7f8e080c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:48.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.441+0000 7f8e07400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8e080cad90 0x7f8e080c4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60658/0 (socket says 172.21.15.138:60658) 2024-01-21T22:52:48.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.441+0000 7f8e07400640 1 -- 172.21.15.138:0/631617870 learned_addr learned my addr 172.21.15.138:0/631617870 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:48.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.444+0000 7f8e07400640 1 -- 172.21.15.138:0/631617870 >> v2:172.21.15.152:3300/0 conn(0x7f8e080cb1d0 msgr2=0x7f8e080c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:48.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.444+0000 7f8e07400640 1 --2- 172.21.15.138:0/631617870 >> v2:172.21.15.152:3300/0 conn(0x7f8e080cb1d0 0x7f8e080c4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:48.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.444+0000 7f8e07400640 1 -- 172.21.15.138:0/631617870 >> v1:172.21.15.152:6789/0 conn(0x7f8e08153d30 legacy=0x7f8e0815a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:48.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.444+0000 7f8e07400640 1 -- 172.21.15.138:0/631617870 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e080c4450 con 0x7f8e080cad90 2024-01-21T22:52:48.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.446+0000 7f8e07400640 1 --2- 172.21.15.138:0/631617870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e080cad90 0x7f8e080c4a20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8e080ca580 tx=0x7f8df002b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967fc0cbe09884e3 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:48.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.450+0000 7f8e06a00640 1 -- 172.21.15.138:0/631617870 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df0030880 con 0x7f8e080cad90 2024-01-21T22:52:48.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.450+0000 7f8e06a00640 1 -- 172.21.15.138:0/631617870 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df00309e0 con 0x7f8e080cad90 2024-01-21T22:52:48.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.450+0000 7f8e06a00640 1 -- 172.21.15.138:0/631617870 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df0030d70 con 0x7f8e080cad90 2024-01-21T22:52:48.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.451+0000 7f8e0f200640 1 -- 172.21.15.138:0/631617870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e080cad90 msgr2=0x7f8e080c4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:48.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.451+0000 7f8e0f200640 1 --2- 172.21.15.138:0/631617870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e080cad90 0x7f8e080c4a20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8e080ca580 tx=0x7f8df002b9c0 comp rx=0 tx=0).stop 2024-01-21T22:52:48.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.451+0000 7f8e0f200640 1 -- 172.21.15.138:0/631617870 shutdown_connections 2024-01-21T22:52:48.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.451+0000 7f8e0f200640 1 --2- 172.21.15.138:0/631617870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e080cad90 0x7f8e080c4a20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:48.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.451+0000 7f8e0f200640 1 --2- 172.21.15.138:0/631617870 >> v2:172.21.15.152:3300/0 conn(0x7f8e080cb1d0 0x7f8e080c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:48.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.451+0000 7f8e0f200640 1 -- 172.21.15.138:0/631617870 >> 172.21.15.138:0/631617870 conn(0x7f8e0806cee0 msgr2=0x7f8e08066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:48.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.451+0000 7f8e0f200640 1 -- 172.21.15.138:0/631617870 shutdown_connections 2024-01-21T22:52:48.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.451+0000 7f8e0f200640 1 -- 172.21.15.138:0/631617870 wait complete. 2024-01-21T22:52:48.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.453+0000 7f8e0f200640 1 Processor -- start 2024-01-21T22:52:48.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.453+0000 7f8e0f200640 1 -- start start 2024-01-21T22:52:48.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.454+0000 7f8e0f200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8e080cad90 0x7f8e08150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:48.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.454+0000 7f8e0f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e080cb1d0 0x7f8e081f1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:48.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.454+0000 7f8e0f200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e08153d30 0x7f8e081f2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:48.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.454+0000 7f8e0f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0814b160 con 0x7f8e080cb1d0 2024-01-21T22:52:48.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.454+0000 7f8e0f200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e081f1930 con 0x7f8e08153d30 2024-01-21T22:52:48.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.454+0000 7f8e0f200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0814c010 con 0x7f8e080cad90 2024-01-21T22:52:48.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.454+0000 7f8e07e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8e080cad90 0x7f8e08150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:48.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.454+0000 7f8e07400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e080cb1d0 0x7f8e081f1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:48.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.454+0000 7f8e0d200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e08153d30 0x7f8e081f2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:48.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.455+0000 7f8e07400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e080cb1d0 0x7f8e081f1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60666/0 (socket says 172.21.15.138:60666) 2024-01-21T22:52:48.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.455+0000 7f8e07400640 1 -- 172.21.15.138:0/4091285986 learned_addr learned my addr 172.21.15.138:0/4091285986 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:48.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.458+0000 7f8e0d200640 1 -- 172.21.15.138:0/4091285986 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8e080cad90 msgr2=0x7f8e08150cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:48.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.458+0000 7f8e0d200640 1 --2- 172.21.15.138:0/4091285986 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8e080cad90 0x7f8e08150cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:48.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.458+0000 7f8e0d200640 1 -- 172.21.15.138:0/4091285986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e080cb1d0 msgr2=0x7f8e081f1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:48.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.458+0000 7f8e0d200640 1 --2- 172.21.15.138:0/4091285986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e080cb1d0 0x7f8e081f1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:48.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.458+0000 7f8e0d200640 1 -- 172.21.15.138:0/4091285986 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df00094d0 con 0x7f8e08153d30 2024-01-21T22:52:48.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.460+0000 7f8e0d200640 1 --2- 172.21.15.138:0/4091285986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e08153d30 0x7f8e081f2300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8df800e7f0 tx=0x7f8df800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:48.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.463+0000 7f8dffe00640 1 -- 172.21.15.138:0/4091285986 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8009590 con 0x7f8e08153d30 2024-01-21T22:52:48.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.463+0000 7f8e0f200640 1 -- 172.21.15.138:0/4091285986 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e080c9800 con 0x7f8e08153d30 2024-01-21T22:52:48.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.463+0000 7f8e0f200640 1 -- 172.21.15.138:0/4091285986 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e081f1000 con 0x7f8e08153d30 2024-01-21T22:52:48.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.464+0000 7f8e0f200640 1 -- 172.21.15.138:0/4091285986 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8e0806b080 con 0x7f8e08153d30 2024-01-21T22:52:48.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.464+0000 7f8dffe00640 1 -- 172.21.15.138:0/4091285986 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df8004020 con 0x7f8e08153d30 2024-01-21T22:52:48.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.464+0000 7f8dffe00640 1 -- 172.21.15.138:0/4091285986 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df8004390 con 0x7f8e08153d30 2024-01-21T22:52:48.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.486+0000 7f8dffe00640 1 -- 172.21.15.138:0/4091285986 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f8df800c940 con 0x7f8e08153d30 2024-01-21T22:52:48.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.487+0000 7f8dffe00640 1 --2- 172.21.15.138:0/4091285986 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f8de006b330 0x7f8de006d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:48.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.487+0000 7f8e07e00640 1 --2- 172.21.15.138:0/4091285986 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f8de006b330 0x7f8de006d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:48.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.488+0000 7f8e07e00640 1 --2- 172.21.15.138:0/4091285986 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f8de006b330 0x7f8de006d7f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8e000012e0 tx=0x7f8e00009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:48.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.489+0000 7f8dffe00640 1 -- 172.21.15.138:0/4091285986 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8df8058960 con 0x7f8e08153d30 2024-01-21T22:52:48.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.489+0000 7f8dffe00640 1 --2- 172.21.15.138:0/4091285986 >> [v2:172.21.15.138:6800/1365838748,v1:172.21.15.138:6801/1365838748] conn(0x7f8de0073530 0x7f8de0075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:48.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.489+0000 7f8dffe00640 1 -- 172.21.15.138:0/4091285986 --> [v2:172.21.15.138:6800/1365838748,v1:172.21.15.138:6801/1365838748] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8de0076020 con 0x7f8de0073530 2024-01-21T22:52:48.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.489+0000 7f8dffe00640 1 -- 172.21.15.138:0/4091285986 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8df80044f0 con 0x7f8e08153d30 2024-01-21T22:52:48.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.489+0000 7f8e07400640 1 --2- 172.21.15.138:0/4091285986 >> [v2:172.21.15.138:6800/1365838748,v1:172.21.15.138:6801/1365838748] conn(0x7f8de0073530 0x7f8de0075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:48.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.490+0000 7f8e07400640 1 --2- 172.21.15.138:0/4091285986 >> [v2:172.21.15.138:6800/1365838748,v1:172.21.15.138:6801/1365838748] conn(0x7f8de0073530 0x7f8de0075950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:48.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.492+0000 7f8dffe00640 1 -- 172.21.15.138:0/4091285986 <== osd.2 v2:172.21.15.138:6800/1365838748 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8de0076020 con 0x7f8de0073530 2024-01-21T22:52:48.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.508+0000 7f8e0f200640 1 -- 172.21.15.138:0/4091285986 --> [v2:172.21.15.138:6800/1365838748,v1:172.21.15.138:6801/1365838748] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8e0806ada0 con 0x7f8de0073530 2024-01-21T22:52:48.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.508+0000 7f8dffe00640 1 -- 172.21.15.138:0/4091285986 <== osd.2 v2:172.21.15.138:6800/1365838748 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8df0030b50 con 0x7f8de0073530 2024-01-21T22:52:48.510 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-01-21T22:52:48.510 INFO:teuthology.orchestra.run.smithi138.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-21T22:52:48.510 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-01-21T22:52:48.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.509+0000 7f8e0f200640 1 -- 172.21.15.138:0/4091285986 >> [v2:172.21.15.138:6800/1365838748,v1:172.21.15.138:6801/1365838748] conn(0x7f8de0073530 msgr2=0x7f8de0075950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:48.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.509+0000 7f8e0f200640 1 --2- 172.21.15.138:0/4091285986 >> [v2:172.21.15.138:6800/1365838748,v1:172.21.15.138:6801/1365838748] conn(0x7f8de0073530 0x7f8de0075950 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:48.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.509+0000 7f8e0f200640 1 -- 172.21.15.138:0/4091285986 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f8de006b330 msgr2=0x7f8de006d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:48.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.509+0000 7f8e0f200640 1 --2- 172.21.15.138:0/4091285986 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f8de006b330 0x7f8de006d7f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8e000012e0 tx=0x7f8e00009040 comp rx=0 tx=0).stop 2024-01-21T22:52:48.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.510+0000 7f8e0f200640 1 -- 172.21.15.138:0/4091285986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e08153d30 msgr2=0x7f8e081f2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:48.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.510+0000 7f8e0f200640 1 --2- 172.21.15.138:0/4091285986 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8e08153d30 0x7f8e081f2300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8df800e7f0 tx=0x7f8df800ecb0 comp rx=0 tx=0).stop 2024-01-21T22:52:48.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.510+0000 7f8e0d200640 1 -- 172.21.15.138:0/4091285986 reap_dead start 2024-01-21T22:52:48.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.510+0000 7f8e0f200640 1 -- 172.21.15.138:0/4091285986 shutdown_connections 2024-01-21T22:52:48.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.510+0000 7f8e0f200640 1 -- 172.21.15.138:0/4091285986 >> 172.21.15.138:0/4091285986 conn(0x7f8e0806cee0 msgr2=0x7f8e080664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:48.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.510+0000 7f8e0f200640 1 -- 172.21.15.138:0/4091285986 shutdown_connections 2024-01-21T22:52:48.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.510+0000 7f8e0f200640 1 -- 172.21.15.138:0/4091285986 wait complete. 2024-01-21T22:52:48.532 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-01-21T22:52:48.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.737+0000 7f514be00640 1 Processor -- start 2024-01-21T22:52:48.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.737+0000 7f514be00640 1 -- start start 2024-01-21T22:52:48.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.739+0000 7f514be00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f514c0c09e0 0x7f514c0c6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:48.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.739+0000 7f514be00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f514c159e70 con 0x7f514c0c0490 2024-01-21T22:52:48.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.739+0000 7f514be00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f514c15a0d0 con 0x7f514c06bba0 2024-01-21T22:52:48.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.739+0000 7f514be00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f514c15a370 con 0x7f514c0c09e0 2024-01-21T22:52:48.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.739+0000 7f514b400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f514c0c09e0 0x7f514c0c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:48.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.739+0000 7f5141e00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f514c0c0490 0x7f514c1620d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42464/0 (socket says 172.21.15.138:42464) 2024-01-21T22:52:48.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.739+0000 7f5141e00640 1 -- 172.21.15.138:0/3267355741 learned_addr learned my addr 172.21.15.138:0/3267355741 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:52:48.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.742+0000 7f514a000640 1 -- 172.21.15.138:0/3267355741 <== mon.0 v1:172.21.15.138:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1571273619 0 0) 0x7f514c159e70 con 0x7f514c0c0490 2024-01-21T22:52:48.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.743+0000 7f514a000640 1 -- 172.21.15.138:0/3267355741 --> v1:172.21.15.138:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f512c003520 con 0x7f514c0c0490 2024-01-21T22:52:48.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.743+0000 7f514a000640 1 -- 172.21.15.138:0/3267355741 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4026594719 0 0) 0x7f514c15a0d0 con 0x7f514c06bba0 2024-01-21T22:52:48.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.743+0000 7f514a000640 1 -- 172.21.15.138:0/3267355741 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f514c159e70 con 0x7f514c06bba0 2024-01-21T22:52:48.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.745+0000 7f514b400640 1 -- 172.21.15.138:0/3267355741 >> v1:172.21.15.152:6789/0 conn(0x7f514c06bba0 legacy=0x7f514c159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:48.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.745+0000 7f514b400640 1 -- 172.21.15.138:0/3267355741 >> v1:172.21.15.138:6789/0 conn(0x7f514c0c0490 legacy=0x7f514c1620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:48.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.745+0000 7f514b400640 1 -- 172.21.15.138:0/3267355741 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c15a650 con 0x7f514c0c09e0 2024-01-21T22:52:48.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.748+0000 7f514b400640 1 --2- 172.21.15.138:0/3267355741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f514c0c09e0 0x7f514c0c6430 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5144009d90 tx=0x7f514402bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d1812333a45fce2 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:48.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.752+0000 7f514a000640 1 -- 172.21.15.138:0/3267355741 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514402cd80 con 0x7f514c0c09e0 2024-01-21T22:52:48.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.753+0000 7f514a000640 1 -- 172.21.15.138:0/3267355741 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5144034440 con 0x7f514c0c09e0 2024-01-21T22:52:48.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.753+0000 7f514a000640 1 -- 172.21.15.138:0/3267355741 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51440347d0 con 0x7f514c0c09e0 2024-01-21T22:52:48.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.753+0000 7f514be00640 1 -- 172.21.15.138:0/3267355741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f514c0c09e0 msgr2=0x7f514c0c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:48.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.753+0000 7f514be00640 1 --2- 172.21.15.138:0/3267355741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f514c0c09e0 0x7f514c0c6430 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5144009d90 tx=0x7f514402bfd0 comp rx=0 tx=0).stop 2024-01-21T22:52:48.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.753+0000 7f514be00640 1 -- 172.21.15.138:0/3267355741 shutdown_connections 2024-01-21T22:52:48.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.753+0000 7f514be00640 1 --2- 172.21.15.138:0/3267355741 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f514c0c09e0 0x7f514c0c6430 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:48.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.753+0000 7f514be00640 1 -- 172.21.15.138:0/3267355741 >> 172.21.15.138:0/3267355741 conn(0x7f514c06d930 msgr2=0x7f514c06dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:48.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.754+0000 7f514be00640 1 -- 172.21.15.138:0/3267355741 shutdown_connections 2024-01-21T22:52:48.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.754+0000 7f514be00640 1 -- 172.21.15.138:0/3267355741 wait complete. 2024-01-21T22:52:48.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.755+0000 7f514be00640 1 Processor -- start 2024-01-21T22:52:48.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.756+0000 7f514be00640 1 -- start start 2024-01-21T22:52:48.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.756+0000 7f514be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f514c06bba0 0x7f514c0af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:48.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.756+0000 7f514be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f514c0c0490 0x7f514c14a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:48.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.757+0000 7f514be00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f514c0c09e0 0x7f514c14aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:48.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.757+0000 7f514be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c0b04c0 con 0x7f514c06bba0 2024-01-21T22:52:48.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.757+0000 7f514be00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c0af720 con 0x7f514c0c0490 2024-01-21T22:52:48.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.757+0000 7f514be00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514c1f0750 con 0x7f514c0c09e0 2024-01-21T22:52:48.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.757+0000 7f5141e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f514c0c0490 0x7f514c14a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:48.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.757+0000 7f514b400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f514c0c09e0 0x7f514c14aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:48.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.758+0000 7f5141e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f514c0c0490 0x7f514c14a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:39154/0 (socket says 172.21.15.138:39154) 2024-01-21T22:52:48.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.758+0000 7f5141e00640 1 -- 172.21.15.138:0/3830794133 learned_addr learned my addr 172.21.15.138:0/3830794133 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:48.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.761+0000 7f514b400640 1 -- 172.21.15.138:0/3830794133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f514c0c0490 msgr2=0x7f514c14a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:48.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.761+0000 7f514b400640 1 --2- 172.21.15.138:0/3830794133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f514c0c0490 0x7f514c14a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:48.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.761+0000 7f514b400640 1 -- 172.21.15.138:0/3830794133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f514c06bba0 msgr2=0x7f514c0af1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:48.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.761+0000 7f514b400640 1 --2- 172.21.15.138:0/3830794133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f514c06bba0 0x7f514c0af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:48.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.761+0000 7f514b400640 1 -- 172.21.15.138:0/3830794133 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5144009a70 con 0x7f514c0c09e0 2024-01-21T22:52:48.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.761+0000 7f5141e00640 1 --2- 172.21.15.138:0/3830794133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f514c0c0490 0x7f514c14a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-21T22:52:48.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.765+0000 7f514b400640 1 --2- 172.21.15.138:0/3830794133 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f514c0c09e0 0x7f514c14aa00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f514c15a000 tx=0x7f5144004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:48.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.771+0000 7f5143400640 1 -- 172.21.15.138:0/3830794133 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514403e410 con 0x7f514c0c09e0 2024-01-21T22:52:48.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.772+0000 7f5143400640 1 -- 172.21.15.138:0/3830794133 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514403e5b0 con 0x7f514c0c09e0 2024-01-21T22:52:48.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.772+0000 7f5143400640 1 -- 172.21.15.138:0/3830794133 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514403e960 con 0x7f514c0c09e0 2024-01-21T22:52:48.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.772+0000 7f514be00640 1 -- 172.21.15.138:0/3830794133 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514c0af940 con 0x7f514c0c09e0 2024-01-21T22:52:48.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.772+0000 7f514be00640 1 -- 172.21.15.138:0/3830794133 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514c0aff00 con 0x7f514c0c09e0 2024-01-21T22:52:48.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.773+0000 7f514be00640 1 -- 172.21.15.138:0/3830794133 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514c1f05b0 con 0x7f514c0c09e0 2024-01-21T22:52:48.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.793+0000 7f5143400640 1 -- 172.21.15.138:0/3830794133 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f514403b050 con 0x7f514c0c09e0 2024-01-21T22:52:48.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.794+0000 7f5143400640 1 --2- 172.21.15.138:0/3830794133 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f512406b3b0 0x7f512406d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:48.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.795+0000 7f514aa00640 1 --2- 172.21.15.138:0/3830794133 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f512406b3b0 0x7f512406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:48.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:48.795+0000 7f514aa00640 1 --2- 172.21.15.138:0/3830794133 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f512406b3b0 0x7f512406d870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f513c0010e0 tx=0x7f513c007950 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:49.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.024+0000 7f5143400640 1 -- 172.21.15.138:0/3830794133 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f514407b000 con 0x7f514c0c09e0 2024-01-21T22:52:49.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.035+0000 7f5143400640 1 -- 172.21.15.138:0/3830794133 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f5144077340 con 0x7f514c0c09e0 2024-01-21T22:52:49.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.244+0000 7f514be00640 1 -- 172.21.15.138:0/3830794133 --> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f514c15dd10 con 0x7f512406b3b0 2024-01-21T22:52:49.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.245+0000 7f5143400640 1 -- 172.21.15.138:0/3830794133 <== mgr.4111 v2:172.21.15.152:6800/314625564 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f514c15dd10 con 0x7f512406b3b0 2024-01-21T22:52:49.246 INFO:teuthology.orchestra.run.smithi138.stderr:instructed osd(s) 2 to deep-scrub 2024-01-21T22:52:49.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.248+0000 7f514be00640 1 -- 172.21.15.138:0/3830794133 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f512406b3b0 msgr2=0x7f512406d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:49.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.248+0000 7f514be00640 1 --2- 172.21.15.138:0/3830794133 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f512406b3b0 0x7f512406d870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f513c0010e0 tx=0x7f513c007950 comp rx=0 tx=0).stop 2024-01-21T22:52:49.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.248+0000 7f514be00640 1 -- 172.21.15.138:0/3830794133 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f514c0c09e0 msgr2=0x7f514c14aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:49.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.248+0000 7f514be00640 1 --2- 172.21.15.138:0/3830794133 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f514c0c09e0 0x7f514c14aa00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f514c15a000 tx=0x7f5144004a40 comp rx=0 tx=0).stop 2024-01-21T22:52:49.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.248+0000 7f514be00640 1 -- 172.21.15.138:0/3830794133 shutdown_connections 2024-01-21T22:52:49.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.248+0000 7f514be00640 1 --2- 172.21.15.138:0/3830794133 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f512406b3b0 0x7f512406d870 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:49.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.248+0000 7f514be00640 1 --2- 172.21.15.138:0/3830794133 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f514c0c09e0 0x7f514c14aa00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:49.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.248+0000 7f514be00640 1 --2- 172.21.15.138:0/3830794133 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f514c0c0490 0x7f514c14a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:49.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.248+0000 7f514be00640 1 --2- 172.21.15.138:0/3830794133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f514c06bba0 0x7f514c0af1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:49.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.248+0000 7f514be00640 1 -- 172.21.15.138:0/3830794133 >> 172.21.15.138:0/3830794133 conn(0x7f514c06d930 msgr2=0x7f514c06dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:49.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.248+0000 7f514be00640 1 -- 172.21.15.138:0/3830794133 shutdown_connections 2024-01-21T22:52:49.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.248+0000 7f514be00640 1 -- 172.21.15.138:0/3830794133 wait complete. 2024-01-21T22:52:49.268 INFO:tasks.ceph:Scrubbing osd.3 2024-01-21T22:52:49.268 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-01-21T22:52:49.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.466+0000 7f70dd800640 1 Processor -- start 2024-01-21T22:52:49.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.467+0000 7f70dd800640 1 -- start start 2024-01-21T22:52:49.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.468+0000 7f70dd800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70d806d2b0 0x7f70d806c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:49.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.468+0000 7f70dd800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f70d815b460 0x7f70d815b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:49.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.468+0000 7f70dd800640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d815afd0 con 0x7f70d8151c00 2024-01-21T22:52:49.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.468+0000 7f70dd800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d806eb00 con 0x7f70d815b460 2024-01-21T22:52:49.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.468+0000 7f70dd800640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d806d6f0 con 0x7f70d806d2b0 2024-01-21T22:52:49.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.469+0000 7f70cfe00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70d806d2b0 0x7f70d806c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:49.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.469+0000 7f70d6a00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f70d8151c00 0x7f70d8072400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:34260/0 (socket says 172.21.15.138:34260) 2024-01-21T22:52:49.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.469+0000 7f70d6a00640 1 -- 172.21.15.138:0/2138822289 learned_addr learned my addr 172.21.15.138:0/2138822289 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:52:49.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.469+0000 7f70d7400640 1 --2- 172.21.15.138:0/2138822289 >> v2:172.21.15.138:3300/0 conn(0x7f70d815b460 0x7f70d815b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:49.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.472+0000 7f70cfe00640 1 -- 172.21.15.138:0/2138822289 >> v2:172.21.15.138:3300/0 conn(0x7f70d815b460 msgr2=0x7f70d815b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:49.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.472+0000 7f70cfe00640 1 --2- 172.21.15.138:0/2138822289 >> v2:172.21.15.138:3300/0 conn(0x7f70d815b460 0x7f70d815b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:49.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.472+0000 7f70cfe00640 1 -- 172.21.15.138:0/2138822289 >> v1:172.21.15.152:6789/0 conn(0x7f70d8151c00 legacy=0x7f70d8072400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:49.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.472+0000 7f70cfe00640 1 -- 172.21.15.138:0/2138822289 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d80c5be0 con 0x7f70d806d2b0 2024-01-21T22:52:49.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.474+0000 7f70cfe00640 1 --2- 172.21.15.138:0/2138822289 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70d806d2b0 0x7f70d806c9d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f70c00097a0 tx=0x7f70c002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=100aa41115087205 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:49.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.478+0000 7f70d6000640 1 -- 172.21.15.138:0/2138822289 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c002d040 con 0x7f70d806d2b0 2024-01-21T22:52:49.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.478+0000 7f70d6000640 1 -- 172.21.15.138:0/2138822289 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c0031810 con 0x7f70d806d2b0 2024-01-21T22:52:49.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.478+0000 7f70d6000640 1 -- 172.21.15.138:0/2138822289 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c0031ba0 con 0x7f70d806d2b0 2024-01-21T22:52:49.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.478+0000 7f70dd800640 1 -- 172.21.15.138:0/2138822289 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70d806d2b0 msgr2=0x7f70d806c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:49.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.478+0000 7f70dd800640 1 --2- 172.21.15.138:0/2138822289 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70d806d2b0 0x7f70d806c9d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f70c00097a0 tx=0x7f70c002b8b0 comp rx=0 tx=0).stop 2024-01-21T22:52:49.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.479+0000 7f70dd800640 1 -- 172.21.15.138:0/2138822289 shutdown_connections 2024-01-21T22:52:49.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.479+0000 7f70dd800640 1 --2- 172.21.15.138:0/2138822289 >> v2:172.21.15.138:3300/0 conn(0x7f70d815b460 0x7f70d815b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:49.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.479+0000 7f70dd800640 1 --2- 172.21.15.138:0/2138822289 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70d806d2b0 0x7f70d806c9d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:49.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.479+0000 7f70dd800640 1 -- 172.21.15.138:0/2138822289 >> 172.21.15.138:0/2138822289 conn(0x7f70d806add0 msgr2=0x7f70d80caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:49.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.479+0000 7f70dd800640 1 -- 172.21.15.138:0/2138822289 shutdown_connections 2024-01-21T22:52:49.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.479+0000 7f70dd800640 1 -- 172.21.15.138:0/2138822289 wait complete. 2024-01-21T22:52:49.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.480+0000 7f70dd800640 1 Processor -- start 2024-01-21T22:52:49.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.481+0000 7f70dd800640 1 -- start start 2024-01-21T22:52:49.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.481+0000 7f70dd800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70d806d2b0 0x7f70d8159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:49.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.481+0000 7f70dd800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70d8151c00 0x7f70d80c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:49.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.481+0000 7f70dd800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70d815b460 0x7f70d80c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:49.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.481+0000 7f70dd800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d81592a0 con 0x7f70d815b460 2024-01-21T22:52:49.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.481+0000 7f70dd800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d8158a20 con 0x7f70d806d2b0 2024-01-21T22:52:49.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.481+0000 7f70dd800640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d8158ce0 con 0x7f70d8151c00 2024-01-21T22:52:49.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.482+0000 7f70d7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70d815b460 0x7f70d80c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:49.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.482+0000 7f70d6a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70d806d2b0 0x7f70d8159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:49.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.482+0000 7f70cfe00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70d8151c00 0x7f70d80c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:49.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.483+0000 7f70d7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70d815b460 0x7f70d80c9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60690/0 (socket says 172.21.15.138:60690) 2024-01-21T22:52:49.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.483+0000 7f70d7400640 1 -- 172.21.15.138:0/2666552469 learned_addr learned my addr 172.21.15.138:0/2666552469 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:49.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.487+0000 7f70d7400640 1 -- 172.21.15.138:0/2666552469 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70d8151c00 msgr2=0x7f70d80c61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:49.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.487+0000 7f70d7400640 1 --2- 172.21.15.138:0/2666552469 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f70d8151c00 0x7f70d80c61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:49.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.487+0000 7f70d7400640 1 -- 172.21.15.138:0/2666552469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70d806d2b0 msgr2=0x7f70d8159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:49.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.487+0000 7f70d7400640 1 --2- 172.21.15.138:0/2666552469 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f70d806d2b0 0x7f70d8159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:49.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.487+0000 7f70d7400640 1 -- 172.21.15.138:0/2666552469 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c0009400 con 0x7f70d815b460 2024-01-21T22:52:49.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.490+0000 7f70d7400640 1 --2- 172.21.15.138:0/2666552469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70d815b460 0x7f70d80c9830 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f70d000efc0 tx=0x7f70d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:49.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.492+0000 7f70cea00640 1 -- 172.21.15.138:0/2666552469 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d0009ba0 con 0x7f70d815b460 2024-01-21T22:52:49.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.492+0000 7f70cea00640 1 -- 172.21.15.138:0/2666552469 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d000f040 con 0x7f70d815b460 2024-01-21T22:52:49.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.492+0000 7f70cea00640 1 -- 172.21.15.138:0/2666552469 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d00146c0 con 0x7f70d815b460 2024-01-21T22:52:49.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.492+0000 7f70dd800640 1 -- 172.21.15.138:0/2666552469 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d8158fc0 con 0x7f70d815b460 2024-01-21T22:52:49.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.493+0000 7f70dd800640 1 -- 172.21.15.138:0/2666552469 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d80c9310 con 0x7f70d815b460 2024-01-21T22:52:49.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.494+0000 7f70dd800640 1 -- 172.21.15.138:0/2666552469 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f70d80c56f0 con 0x7f70d815b460 2024-01-21T22:52:49.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.514+0000 7f70cea00640 1 -- 172.21.15.138:0/2666552469 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f70d0014820 con 0x7f70d815b460 2024-01-21T22:52:49.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.515+0000 7f70cea00640 1 --2- 172.21.15.138:0/2666552469 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f70b006b3b0 0x7f70b006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:49.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.516+0000 7f70d6a00640 1 --2- 172.21.15.138:0/2666552469 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f70b006b3b0 0x7f70b006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:49.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.517+0000 7f70d6a00640 1 --2- 172.21.15.138:0/2666552469 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f70b006b3b0 0x7f70b006d870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f70c8007fd0 tx=0x7f70c8007e80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:49.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.518+0000 7f70cea00640 1 -- 172.21.15.138:0/2666552469 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f70d0059720 con 0x7f70d815b460 2024-01-21T22:52:49.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.518+0000 7f70cea00640 1 --2- 172.21.15.138:0/2666552469 >> [v2:172.21.15.138:6806/1841873885,v1:172.21.15.138:6808/1841873885] conn(0x7f70b00735b0 0x7f70b00759d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:49.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.518+0000 7f70cea00640 1 -- 172.21.15.138:0/2666552469 --> [v2:172.21.15.138:6806/1841873885,v1:172.21.15.138:6808/1841873885] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f70b00760a0 con 0x7f70b00735b0 2024-01-21T22:52:49.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.518+0000 7f70cea00640 1 -- 172.21.15.138:0/2666552469 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70d0055a60 con 0x7f70d815b460 2024-01-21T22:52:49.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.518+0000 7f70cfe00640 1 --2- 172.21.15.138:0/2666552469 >> [v2:172.21.15.138:6806/1841873885,v1:172.21.15.138:6808/1841873885] conn(0x7f70b00735b0 0x7f70b00759d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:49.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.519+0000 7f70cfe00640 1 --2- 172.21.15.138:0/2666552469 >> [v2:172.21.15.138:6806/1841873885,v1:172.21.15.138:6808/1841873885] conn(0x7f70b00735b0 0x7f70b00759d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:49.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.520+0000 7f70cea00640 1 -- 172.21.15.138:0/2666552469 <== osd.3 v2:172.21.15.138:6806/1841873885 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f70b00760a0 con 0x7f70b00735b0 2024-01-21T22:52:49.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.535+0000 7f70dd800640 1 -- 172.21.15.138:0/2666552469 --> [v2:172.21.15.138:6806/1841873885,v1:172.21.15.138:6808/1841873885] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f70d80c2910 con 0x7f70b00735b0 2024-01-21T22:52:49.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.536+0000 7f70cea00640 1 -- 172.21.15.138:0/2666552469 <== osd.3 v2:172.21.15.138:6806/1841873885 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f70c0031a80 con 0x7f70b00735b0 2024-01-21T22:52:49.537 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-01-21T22:52:49.537 INFO:teuthology.orchestra.run.smithi138.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-21T22:52:49.538 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-01-21T22:52:49.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.536+0000 7f70dd800640 1 -- 172.21.15.138:0/2666552469 >> [v2:172.21.15.138:6806/1841873885,v1:172.21.15.138:6808/1841873885] conn(0x7f70b00735b0 msgr2=0x7f70b00759d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:49.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.536+0000 7f70dd800640 1 --2- 172.21.15.138:0/2666552469 >> [v2:172.21.15.138:6806/1841873885,v1:172.21.15.138:6808/1841873885] conn(0x7f70b00735b0 0x7f70b00759d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:49.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.537+0000 7f70dd800640 1 -- 172.21.15.138:0/2666552469 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f70b006b3b0 msgr2=0x7f70b006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:49.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.537+0000 7f70dd800640 1 --2- 172.21.15.138:0/2666552469 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f70b006b3b0 0x7f70b006d870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f70c8007fd0 tx=0x7f70c8007e80 comp rx=0 tx=0).stop 2024-01-21T22:52:49.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.537+0000 7f70dd800640 1 -- 172.21.15.138:0/2666552469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70d815b460 msgr2=0x7f70d80c9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:49.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.537+0000 7f70dd800640 1 --2- 172.21.15.138:0/2666552469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70d815b460 0x7f70d80c9830 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f70d000efc0 tx=0x7f70d000c490 comp rx=0 tx=0).stop 2024-01-21T22:52:49.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.537+0000 7f70d7400640 1 -- 172.21.15.138:0/2666552469 reap_dead start 2024-01-21T22:52:49.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.537+0000 7f70dd800640 1 -- 172.21.15.138:0/2666552469 shutdown_connections 2024-01-21T22:52:49.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.537+0000 7f70dd800640 1 -- 172.21.15.138:0/2666552469 >> 172.21.15.138:0/2666552469 conn(0x7f70d806add0 msgr2=0x7f70d80c7230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:49.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.537+0000 7f70dd800640 1 -- 172.21.15.138:0/2666552469 shutdown_connections 2024-01-21T22:52:49.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.537+0000 7f70dd800640 1 -- 172.21.15.138:0/2666552469 wait complete. 2024-01-21T22:52:49.559 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-01-21T22:52:49.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.762+0000 7f6d99200640 1 Processor -- start 2024-01-21T22:52:49.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.762+0000 7f6d99200640 1 -- start start 2024-01-21T22:52:49.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.764+0000 7f6d99200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6d9406b9b0 0x7f6d94158280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:49.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.764+0000 7f6d99200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6d940c5970 0x7f6d94161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:49.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.764+0000 7f6d99200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d9406be80 con 0x7f6d94157ea0 2024-01-21T22:52:49.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.764+0000 7f6d99200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d9415dfe0 con 0x7f6d940c5970 2024-01-21T22:52:49.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.764+0000 7f6d99200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9415e2c0 con 0x7f6d9406b9b0 2024-01-21T22:52:49.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.765+0000 7f6d92800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6d940c5970 0x7f6d94161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:49.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.765+0000 7f6d91e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f6d94157ea0 0x7f6d9406c360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:34268/0 (socket says 172.21.15.138:34268) 2024-01-21T22:52:49.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.765+0000 7f6d91e00640 1 -- 172.21.15.138:0/2884324461 learned_addr learned my addr 172.21.15.138:0/2884324461 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:52:49.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.766+0000 7f6d91400640 1 --2- 172.21.15.138:0/2884324461 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6d9406b9b0 0x7f6d94158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:49.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.768+0000 7f6d91400640 1 -- 172.21.15.138:0/2884324461 >> v2:172.21.15.152:3300/0 conn(0x7f6d940c5970 msgr2=0x7f6d94161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:49.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.768+0000 7f6d91400640 1 --2- 172.21.15.138:0/2884324461 >> v2:172.21.15.152:3300/0 conn(0x7f6d940c5970 0x7f6d94161e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:49.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.768+0000 7f6d91400640 1 -- 172.21.15.138:0/2884324461 >> v1:172.21.15.152:6789/0 conn(0x7f6d94157ea0 legacy=0x7f6d9406c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:49.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.768+0000 7f6d91400640 1 -- 172.21.15.138:0/2884324461 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9415e5a0 con 0x7f6d9406b9b0 2024-01-21T22:52:49.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.771+0000 7f6d91400640 1 --2- 172.21.15.138:0/2884324461 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6d9406b9b0 0x7f6d94158280 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0097a0 tx=0x7f6d7c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=907411f726c0f61d server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:49.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.778+0000 7f6d90a00640 1 -- 172.21.15.138:0/2884324461 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c031830 con 0x7f6d9406b9b0 2024-01-21T22:52:49.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.778+0000 7f6d90a00640 1 -- 172.21.15.138:0/2884324461 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7c031990 con 0x7f6d9406b9b0 2024-01-21T22:52:49.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.778+0000 7f6d90a00640 1 -- 172.21.15.138:0/2884324461 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c031d20 con 0x7f6d9406b9b0 2024-01-21T22:52:49.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.778+0000 7f6d99200640 1 -- 172.21.15.138:0/2884324461 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6d9406b9b0 msgr2=0x7f6d94158280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:49.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.778+0000 7f6d99200640 1 --2- 172.21.15.138:0/2884324461 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6d9406b9b0 0x7f6d94158280 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0097a0 tx=0x7f6d7c02b8d0 comp rx=0 tx=0).stop 2024-01-21T22:52:49.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.778+0000 7f6d99200640 1 -- 172.21.15.138:0/2884324461 shutdown_connections 2024-01-21T22:52:49.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.778+0000 7f6d99200640 1 --2- 172.21.15.138:0/2884324461 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6d9406b9b0 0x7f6d94158280 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:49.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.778+0000 7f6d99200640 1 --2- 172.21.15.138:0/2884324461 >> v2:172.21.15.152:3300/0 conn(0x7f6d940c5970 0x7f6d94161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:49.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.778+0000 7f6d99200640 1 -- 172.21.15.138:0/2884324461 >> 172.21.15.138:0/2884324461 conn(0x7f6d940cb070 msgr2=0x7f6d940cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:49.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.778+0000 7f6d99200640 1 -- 172.21.15.138:0/2884324461 shutdown_connections 2024-01-21T22:52:49.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.778+0000 7f6d99200640 1 -- 172.21.15.138:0/2884324461 wait complete. 2024-01-21T22:52:49.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.780+0000 7f6d99200640 1 Processor -- start 2024-01-21T22:52:49.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.781+0000 7f6d99200640 1 -- start start 2024-01-21T22:52:49.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.782+0000 7f6d99200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d9406b9b0 0x7f6d941f4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:49.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.782+0000 7f6d99200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6d940c5970 0x7f6d940c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:49.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.782+0000 7f6d99200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d94157ea0 0x7f6d9414d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:49.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.782+0000 7f6d99200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9414eb00 con 0x7f6d9406b9b0 2024-01-21T22:52:49.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.782+0000 7f6d99200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d941f4850 con 0x7f6d94157ea0 2024-01-21T22:52:49.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.782+0000 7f6d99200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d940caeb0 con 0x7f6d940c5970 2024-01-21T22:52:49.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.783+0000 7f6d92800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d94157ea0 0x7f6d9414d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:49.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.783+0000 7f6d91400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6d940c5970 0x7f6d940c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:49.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.783+0000 7f6d91e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d9406b9b0 0x7f6d941f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:49.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.783+0000 7f6d92800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d94157ea0 0x7f6d9414d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:39198/0 (socket says 172.21.15.138:39198) 2024-01-21T22:52:49.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.783+0000 7f6d92800640 1 -- 172.21.15.138:0/815363680 learned_addr learned my addr 172.21.15.138:0/815363680 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:49.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.783+0000 7f6d91400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6d940c5970 0x7f6d940c6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:60128/0 (socket says 172.21.15.138:60128) 2024-01-21T22:52:49.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.786+0000 7f6d91400640 1 -- 172.21.15.138:0/815363680 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d94157ea0 msgr2=0x7f6d9414d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:49.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.786+0000 7f6d91400640 1 --2- 172.21.15.138:0/815363680 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d94157ea0 0x7f6d9414d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:49.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.788+0000 7f6d91400640 1 -- 172.21.15.138:0/815363680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d9406b9b0 msgr2=0x7f6d941f4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:49.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.788+0000 7f6d91400640 1 --2- 172.21.15.138:0/815363680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d9406b9b0 0x7f6d941f4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:49.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.788+0000 7f6d91400640 1 -- 172.21.15.138:0/815363680 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7c009400 con 0x7f6d940c5970 2024-01-21T22:52:49.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.790+0000 7f6d91400640 1 --2- 172.21.15.138:0/815363680 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6d940c5970 0x7f6d940c6960 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c033000 tx=0x7f6d7c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:49.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.794+0000 7f6d8a000640 1 -- 172.21.15.138:0/815363680 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c03b9f0 con 0x7f6d940c5970 2024-01-21T22:52:49.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.794+0000 7f6d8a000640 1 -- 172.21.15.138:0/815363680 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7c03bb90 con 0x7f6d940c5970 2024-01-21T22:52:49.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.794+0000 7f6d8a000640 1 -- 172.21.15.138:0/815363680 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c0433f0 con 0x7f6d940c5970 2024-01-21T22:52:49.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.794+0000 7f6d99200640 1 -- 172.21.15.138:0/815363680 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d941582f0 con 0x7f6d940c5970 2024-01-21T22:52:49.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.794+0000 7f6d99200640 1 -- 172.21.15.138:0/815363680 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d940ca610 con 0x7f6d940c5970 2024-01-21T22:52:49.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.794+0000 7f6d99200640 1 -- 172.21.15.138:0/815363680 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d94067ce0 con 0x7f6d940c5970 2024-01-21T22:52:49.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.815+0000 7f6d8a000640 1 -- 172.21.15.138:0/815363680 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f6d7c043550 con 0x7f6d940c5970 2024-01-21T22:52:49.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.816+0000 7f6d8a000640 1 --2- 172.21.15.138:0/815363680 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6d6c06b380 0x7f6d6c06d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:49.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.816+0000 7f6d91e00640 1 --2- 172.21.15.138:0/815363680 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6d6c06b380 0x7f6d6c06d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:49.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.817+0000 7f6d91e00640 1 --2- 172.21.15.138:0/815363680 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6d6c06b380 0x7f6d6c06d840 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6d840092a0 tx=0x7f6d84009230 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:49.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:49.817+0000 7f6d8a000640 1 -- 172.21.15.138:0/815363680 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f6d7c07a890 con 0x7f6d940c5970 2024-01-21T22:52:50.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.052+0000 7f6d8a000640 1 -- 172.21.15.138:0/815363680 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6d7c076bd0 con 0x7f6d940c5970 2024-01-21T22:52:50.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.260+0000 7f6d99200640 1 -- 172.21.15.138:0/815363680 --> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d9404ea30 con 0x7f6d6c06b380 2024-01-21T22:52:50.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.262+0000 7f6d8a000640 1 -- 172.21.15.138:0/815363680 <== mgr.4111 v2:172.21.15.152:6800/314625564 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6d9404ea30 con 0x7f6d6c06b380 2024-01-21T22:52:50.263 INFO:teuthology.orchestra.run.smithi138.stderr:instructed osd(s) 3 to deep-scrub 2024-01-21T22:52:50.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.265+0000 7f6d99200640 1 -- 172.21.15.138:0/815363680 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6d6c06b380 msgr2=0x7f6d6c06d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:50.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.265+0000 7f6d99200640 1 --2- 172.21.15.138:0/815363680 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6d6c06b380 0x7f6d6c06d840 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6d840092a0 tx=0x7f6d84009230 comp rx=0 tx=0).stop 2024-01-21T22:52:50.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.265+0000 7f6d99200640 1 -- 172.21.15.138:0/815363680 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6d940c5970 msgr2=0x7f6d940c6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:50.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.265+0000 7f6d99200640 1 --2- 172.21.15.138:0/815363680 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6d940c5970 0x7f6d940c6960 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c033000 tx=0x7f6d7c0046c0 comp rx=0 tx=0).stop 2024-01-21T22:52:50.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.265+0000 7f6d99200640 1 -- 172.21.15.138:0/815363680 shutdown_connections 2024-01-21T22:52:50.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.265+0000 7f6d99200640 1 --2- 172.21.15.138:0/815363680 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6d6c06b380 0x7f6d6c06d840 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:50.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.265+0000 7f6d99200640 1 --2- 172.21.15.138:0/815363680 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6d94157ea0 0x7f6d9414d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:50.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.265+0000 7f6d99200640 1 --2- 172.21.15.138:0/815363680 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6d940c5970 0x7f6d940c6960 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:50.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.265+0000 7f6d99200640 1 --2- 172.21.15.138:0/815363680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d9406b9b0 0x7f6d941f4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:50.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.265+0000 7f6d99200640 1 -- 172.21.15.138:0/815363680 >> 172.21.15.138:0/815363680 conn(0x7f6d940cb070 msgr2=0x7f6d9415ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:50.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.265+0000 7f6d99200640 1 -- 172.21.15.138:0/815363680 shutdown_connections 2024-01-21T22:52:50.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.265+0000 7f6d99200640 1 -- 172.21.15.138:0/815363680 wait complete. 2024-01-21T22:52:50.284 INFO:tasks.ceph:Scrubbing osd.4 2024-01-21T22:52:50.285 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-01-21T22:52:50.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.483+0000 7f726e800640 1 Processor -- start 2024-01-21T22:52:50.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.483+0000 7f726e800640 1 -- start start 2024-01-21T22:52:50.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.484+0000 7f726e800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f726806ad80 0x7f726815a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:50.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.484+0000 7f726e800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f726806b160 0x7f726815cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:50.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.484+0000 7f726e800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f72680cc8c0 0x7f72680ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:50.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.484+0000 7f726e800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72680cb350 con 0x7f726806ad80 2024-01-21T22:52:50.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.484+0000 7f726e800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72680cb630 con 0x7f72680cc8c0 2024-01-21T22:52:50.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.484+0000 7f726e800640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72680cb910 con 0x7f726806b160 2024-01-21T22:52:50.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.495+0000 7f726de00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f72680cc8c0 0x7f72680ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:50.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.495+0000 7f726de00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f72680cc8c0 0x7f72680ca8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:39202/0 (socket says 172.21.15.138:39202) 2024-01-21T22:52:50.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.495+0000 7f726de00640 1 -- 172.21.15.138:0/3166158540 learned_addr learned my addr 172.21.15.138:0/3166158540 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:50.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.497+0000 7f726ca00640 1 --2- 172.21.15.138:0/3166158540 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f726806b160 0x7f726815cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:50.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.498+0000 7f726de00640 1 -- 172.21.15.138:0/3166158540 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f726806b160 msgr2=0x7f726815cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:50.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.498+0000 7f726de00640 1 --2- 172.21.15.138:0/3166158540 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f726806b160 0x7f726815cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:50.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.498+0000 7f726de00640 1 -- 172.21.15.138:0/3166158540 >> v2:172.21.15.138:3300/0 conn(0x7f726806ad80 msgr2=0x7f726815a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:50.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.498+0000 7f726de00640 1 --2- 172.21.15.138:0/3166158540 >> v2:172.21.15.138:3300/0 conn(0x7f726806ad80 0x7f726815a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:50.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.498+0000 7f726de00640 1 -- 172.21.15.138:0/3166158540 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7268159ca0 con 0x7f72680cc8c0 2024-01-21T22:52:50.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.501+0000 7f726de00640 1 --2- 172.21.15.138:0/3166158540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72680cc8c0 0x7f72680ca8b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f726808d1c0 tx=0x7f726002b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e159672263a1ef1d server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:50.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.505+0000 7f7267e00640 1 -- 172.21.15.138:0/3166158540 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260030850 con 0x7f72680cc8c0 2024-01-21T22:52:50.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.505+0000 7f7267e00640 1 -- 172.21.15.138:0/3166158540 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72600309b0 con 0x7f72680cc8c0 2024-01-21T22:52:50.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.505+0000 7f7267e00640 1 -- 172.21.15.138:0/3166158540 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260030d40 con 0x7f72680cc8c0 2024-01-21T22:52:50.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.505+0000 7f726e800640 1 -- 172.21.15.138:0/3166158540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72680cc8c0 msgr2=0x7f72680ca8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:50.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.505+0000 7f726e800640 1 --2- 172.21.15.138:0/3166158540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72680cc8c0 0x7f72680ca8b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f726808d1c0 tx=0x7f726002b990 comp rx=0 tx=0).stop 2024-01-21T22:52:50.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.506+0000 7f726e800640 1 -- 172.21.15.138:0/3166158540 shutdown_connections 2024-01-21T22:52:50.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.506+0000 7f726e800640 1 --2- 172.21.15.138:0/3166158540 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f72680cc8c0 0x7f72680ca8b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:50.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.506+0000 7f726e800640 1 --2- 172.21.15.138:0/3166158540 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f726806b160 0x7f726815cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:50.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.506+0000 7f726e800640 1 --2- 172.21.15.138:0/3166158540 >> v2:172.21.15.138:3300/0 conn(0x7f726806ad80 0x7f726815a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:50.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.506+0000 7f726e800640 1 -- 172.21.15.138:0/3166158540 >> 172.21.15.138:0/3166158540 conn(0x7f726806ca40 msgr2=0x7f726806ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:50.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.506+0000 7f726e800640 1 -- 172.21.15.138:0/3166158540 shutdown_connections 2024-01-21T22:52:50.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.506+0000 7f726e800640 1 -- 172.21.15.138:0/3166158540 wait complete. 2024-01-21T22:52:50.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.508+0000 7f726e800640 1 Processor -- start 2024-01-21T22:52:50.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.509+0000 7f726e800640 1 -- start start 2024-01-21T22:52:50.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.510+0000 7f726e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f726806ad80 0x7f72680b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:50.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.510+0000 7f726e800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f726806b160 0x7f72680b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:50.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.510+0000 7f726e800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72680cc8c0 0x7f72681ef570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:50.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.510+0000 7f726e800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72680cfee0 con 0x7f726806ad80 2024-01-21T22:52:50.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.510+0000 7f726e800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726815ccb0 con 0x7f726806b160 2024-01-21T22:52:50.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.510+0000 7f726e800640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726815cf50 con 0x7f72680cc8c0 2024-01-21T22:52:50.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.511+0000 7f726de00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72680cc8c0 0x7f72681ef570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:50.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.511+0000 7f726ca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f726806b160 0x7f72680b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:50.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.511+0000 7f726de00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72680cc8c0 0x7f72681ef570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:60150/0 (socket says 172.21.15.138:60150) 2024-01-21T22:52:50.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.511+0000 7f726de00640 1 -- 172.21.15.138:0/962112937 learned_addr learned my addr 172.21.15.138:0/962112937 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:50.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.514+0000 7f726de00640 1 -- 172.21.15.138:0/962112937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f726806b160 msgr2=0x7f72680b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:50.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.514+0000 7f726de00640 1 --2- 172.21.15.138:0/962112937 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f726806b160 0x7f72680b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:50.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.514+0000 7f726de00640 1 -- 172.21.15.138:0/962112937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f726806ad80 msgr2=0x7f72680b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:50.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.514+0000 7f726de00640 1 --2- 172.21.15.138:0/962112937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f726806ad80 0x7f72680b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:50.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.514+0000 7f726de00640 1 -- 172.21.15.138:0/962112937 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72600094d0 con 0x7f72680cc8c0 2024-01-21T22:52:50.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.517+0000 7f726de00640 1 --2- 172.21.15.138:0/962112937 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72680cc8c0 0x7f72681ef570 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f726002b9d0 tx=0x7f7260004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:50.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.521+0000 7f7265600640 1 -- 172.21.15.138:0/962112937 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260039c90 con 0x7f72680cc8c0 2024-01-21T22:52:50.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.521+0000 7f7265600640 1 -- 172.21.15.138:0/962112937 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7260039df0 con 0x7f72680cc8c0 2024-01-21T22:52:50.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.521+0000 7f7265600640 1 -- 172.21.15.138:0/962112937 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260043640 con 0x7f72680cc8c0 2024-01-21T22:52:50.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.521+0000 7f726e800640 1 -- 172.21.15.138:0/962112937 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72680cf2d0 con 0x7f72680cc8c0 2024-01-21T22:52:50.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.521+0000 7f726e800640 1 -- 172.21.15.138:0/962112937 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72680cf890 con 0x7f72680cc8c0 2024-01-21T22:52:50.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.522+0000 7f726e800640 1 -- 172.21.15.138:0/962112937 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f72681ef1b0 con 0x7f72680cc8c0 2024-01-21T22:52:50.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.544+0000 7f7265600640 1 -- 172.21.15.138:0/962112937 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f726002e070 con 0x7f72680cc8c0 2024-01-21T22:52:50.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.545+0000 7f7265600640 1 --2- 172.21.15.138:0/962112937 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f724006b3b0 0x7f724006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:50.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.546+0000 7f726d400640 1 --2- 172.21.15.138:0/962112937 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f724006b3b0 0x7f724006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:50.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.546+0000 7f7265600640 1 -- 172.21.15.138:0/962112937 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f726003e070 con 0x7f72680cc8c0 2024-01-21T22:52:50.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.546+0000 7f7265600640 1 --2- 172.21.15.138:0/962112937 >> [v2:172.21.15.152:6826/3960812298,v1:172.21.15.152:6827/3960812298] conn(0x7f72400735b0 0x7f72400759d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:50.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.546+0000 7f7265600640 1 -- 172.21.15.138:0/962112937 --> [v2:172.21.15.152:6826/3960812298,v1:172.21.15.152:6827/3960812298] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f72400760a0 con 0x7f72400735b0 2024-01-21T22:52:50.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.546+0000 7f726ca00640 1 --2- 172.21.15.138:0/962112937 >> [v2:172.21.15.152:6826/3960812298,v1:172.21.15.152:6827/3960812298] conn(0x7f72400735b0 0x7f72400759d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:50.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.546+0000 7f726d400640 1 --2- 172.21.15.138:0/962112937 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f724006b3b0 0x7f724006d870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f725800ab90 tx=0x7f7258009250 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:50.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.547+0000 7f726ca00640 1 --2- 172.21.15.138:0/962112937 >> [v2:172.21.15.152:6826/3960812298,v1:172.21.15.152:6827/3960812298] conn(0x7f72400735b0 0x7f72400759d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:50.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.548+0000 7f7265600640 1 -- 172.21.15.138:0/962112937 <== osd.4 v2:172.21.15.152:6826/3960812298 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f72400760a0 con 0x7f72400735b0 2024-01-21T22:52:50.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.549+0000 7f7265600640 1 -- 172.21.15.138:0/962112937 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7260078050 con 0x7f72680cc8c0 2024-01-21T22:52:50.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.565+0000 7f726e800640 1 -- 172.21.15.138:0/962112937 --> [v2:172.21.15.152:6826/3960812298,v1:172.21.15.152:6827/3960812298] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7268159990 con 0x7f72400735b0 2024-01-21T22:52:50.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.566+0000 7f7265600640 1 -- 172.21.15.138:0/962112937 <== osd.4 v2:172.21.15.152:6826/3960812298 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f7250009bd0 con 0x7f72400735b0 2024-01-21T22:52:50.566 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-01-21T22:52:50.566 INFO:teuthology.orchestra.run.smithi138.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-21T22:52:50.566 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-01-21T22:52:50.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.566+0000 7f726e800640 1 -- 172.21.15.138:0/962112937 >> [v2:172.21.15.152:6826/3960812298,v1:172.21.15.152:6827/3960812298] conn(0x7f72400735b0 msgr2=0x7f72400759d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:50.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.566+0000 7f726e800640 1 --2- 172.21.15.138:0/962112937 >> [v2:172.21.15.152:6826/3960812298,v1:172.21.15.152:6827/3960812298] conn(0x7f72400735b0 0x7f72400759d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:50.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.567+0000 7f726e800640 1 -- 172.21.15.138:0/962112937 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f724006b3b0 msgr2=0x7f724006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:50.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.567+0000 7f726e800640 1 --2- 172.21.15.138:0/962112937 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f724006b3b0 0x7f724006d870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f725800ab90 tx=0x7f7258009250 comp rx=0 tx=0).stop 2024-01-21T22:52:50.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.567+0000 7f726e800640 1 -- 172.21.15.138:0/962112937 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72680cc8c0 msgr2=0x7f72681ef570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:50.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.567+0000 7f726e800640 1 --2- 172.21.15.138:0/962112937 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f72680cc8c0 0x7f72681ef570 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f726002b9d0 tx=0x7f7260004730 comp rx=0 tx=0).stop 2024-01-21T22:52:50.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.567+0000 7f726de00640 1 -- 172.21.15.138:0/962112937 reap_dead start 2024-01-21T22:52:50.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.567+0000 7f726e800640 1 -- 172.21.15.138:0/962112937 shutdown_connections 2024-01-21T22:52:50.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.567+0000 7f726e800640 1 -- 172.21.15.138:0/962112937 >> 172.21.15.138:0/962112937 conn(0x7f726806ca40 msgr2=0x7f7268158420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:50.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.567+0000 7f726e800640 1 -- 172.21.15.138:0/962112937 shutdown_connections 2024-01-21T22:52:50.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.567+0000 7f726e800640 1 -- 172.21.15.138:0/962112937 wait complete. 2024-01-21T22:52:50.590 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-01-21T22:52:50.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.797+0000 7fee93200640 1 Processor -- start 2024-01-21T22:52:50.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.797+0000 7fee93200640 1 -- start start 2024-01-21T22:52:50.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.799+0000 7fee93200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee9415cdc0 0x7fee940c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:50.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.799+0000 7fee93200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee940c4900 con 0x7fee940ca7b0 2024-01-21T22:52:50.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.799+0000 7fee93200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee9415dc90 con 0x7fee9415d1a0 2024-01-21T22:52:50.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.799+0000 7fee93200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9406b4b0 con 0x7fee9415cdc0 2024-01-21T22:52:50.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.800+0000 7fee91e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee9415cdc0 0x7fee940c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:50.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.800+0000 7fee92800640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fee9415d1a0 0x7fee94164150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:34282/0 (socket says 172.21.15.138:34282) 2024-01-21T22:52:50.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.800+0000 7fee92800640 1 -- 172.21.15.138:0/220669616 learned_addr learned my addr 172.21.15.138:0/220669616 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:52:50.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.803+0000 7fee91e00640 1 -- 172.21.15.138:0/220669616 >> v1:172.21.15.152:6789/0 conn(0x7fee9415d1a0 legacy=0x7fee94164150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:50.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.803+0000 7fee91e00640 1 -- 172.21.15.138:0/220669616 >> v1:172.21.15.138:6789/0 conn(0x7fee940ca7b0 legacy=0x7fee9406c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:50.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.803+0000 7fee91e00640 1 -- 172.21.15.138:0/220669616 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee940c4010 con 0x7fee9415cdc0 2024-01-21T22:52:50.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.805+0000 7fee91e00640 1 --2- 172.21.15.138:0/220669616 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee9415cdc0 0x7fee940c9ad0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fee840097d0 tx=0x7fee8402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6350db56159e16fa server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:50.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.808+0000 7fee90a00640 1 -- 172.21.15.138:0/220669616 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee84031890 con 0x7fee9415cdc0 2024-01-21T22:52:50.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.809+0000 7fee90a00640 1 -- 172.21.15.138:0/220669616 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee840319f0 con 0x7fee9415cdc0 2024-01-21T22:52:50.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.809+0000 7fee90a00640 1 -- 172.21.15.138:0/220669616 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee84031d80 con 0x7fee9415cdc0 2024-01-21T22:52:50.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.809+0000 7fee93200640 1 -- 172.21.15.138:0/220669616 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee9415cdc0 msgr2=0x7fee940c9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:50.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.809+0000 7fee93200640 1 --2- 172.21.15.138:0/220669616 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee9415cdc0 0x7fee940c9ad0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fee840097d0 tx=0x7fee8402b930 comp rx=0 tx=0).stop 2024-01-21T22:52:50.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.809+0000 7fee93200640 1 -- 172.21.15.138:0/220669616 shutdown_connections 2024-01-21T22:52:50.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.809+0000 7fee93200640 1 --2- 172.21.15.138:0/220669616 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee9415cdc0 0x7fee940c9ad0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:50.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.809+0000 7fee93200640 1 -- 172.21.15.138:0/220669616 >> 172.21.15.138:0/220669616 conn(0x7fee9406d370 msgr2=0x7fee94066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:50.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.810+0000 7fee93200640 1 -- 172.21.15.138:0/220669616 shutdown_connections 2024-01-21T22:52:50.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.810+0000 7fee93200640 1 -- 172.21.15.138:0/220669616 wait complete. 2024-01-21T22:52:50.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.812+0000 7fee93200640 1 Processor -- start 2024-01-21T22:52:50.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.812+0000 7fee93200640 1 -- start start 2024-01-21T22:52:50.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.813+0000 7fee93200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee940ca7b0 0x7fee94151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:50.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.813+0000 7fee93200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee9415cdc0 0x7fee941e2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:50.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.813+0000 7fee93200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee9415d1a0 0x7fee94147950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:50.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.813+0000 7fee93200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9414fef0 con 0x7fee9415d1a0 2024-01-21T22:52:50.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.813+0000 7fee93200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee941f3b50 con 0x7fee940ca7b0 2024-01-21T22:52:50.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.813+0000 7fee93200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9414ee50 con 0x7fee9415cdc0 2024-01-21T22:52:50.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.814+0000 7fee91e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee940ca7b0 0x7fee94151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:50.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.814+0000 7fee91e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee940ca7b0 0x7fee94151cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:39218/0 (socket says 172.21.15.138:39218) 2024-01-21T22:52:50.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.814+0000 7fee91e00640 1 -- 172.21.15.138:0/3737225084 learned_addr learned my addr 172.21.15.138:0/3737225084 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:50.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.816+0000 7fee91400640 1 --2- 172.21.15.138:0/3737225084 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee9415cdc0 0x7fee941e2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:50.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.816+0000 7fee92800640 1 --2- 172.21.15.138:0/3737225084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee9415d1a0 0x7fee94147950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:50.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.817+0000 7fee91e00640 1 -- 172.21.15.138:0/3737225084 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee9415cdc0 msgr2=0x7fee941e2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:50.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.817+0000 7fee91e00640 1 --2- 172.21.15.138:0/3737225084 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee9415cdc0 0x7fee941e2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:50.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.817+0000 7fee91e00640 1 -- 172.21.15.138:0/3737225084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee9415d1a0 msgr2=0x7fee94147950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:50.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.817+0000 7fee91e00640 1 --2- 172.21.15.138:0/3737225084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee9415d1a0 0x7fee94147950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:50.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.818+0000 7fee91e00640 1 -- 172.21.15.138:0/3737225084 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee84009400 con 0x7fee940ca7b0 2024-01-21T22:52:50.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.820+0000 7fee91e00640 1 --2- 172.21.15.138:0/3737225084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee940ca7b0 0x7fee94151cb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fee840098f0 tx=0x7fee840046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:50.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.822+0000 7fee8a000640 1 -- 172.21.15.138:0/3737225084 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee84031830 con 0x7fee940ca7b0 2024-01-21T22:52:50.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.822+0000 7fee93200640 1 -- 172.21.15.138:0/3737225084 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9414f130 con 0x7fee940ca7b0 2024-01-21T22:52:50.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.823+0000 7fee93200640 1 -- 172.21.15.138:0/3737225084 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee9414f6f0 con 0x7fee940ca7b0 2024-01-21T22:52:50.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.824+0000 7fee93200640 1 -- 172.21.15.138:0/3737225084 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee94074ee0 con 0x7fee940ca7b0 2024-01-21T22:52:50.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.824+0000 7fee8a000640 1 -- 172.21.15.138:0/3737225084 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee84045450 con 0x7fee940ca7b0 2024-01-21T22:52:50.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:50.824+0000 7fee8a000640 1 -- 172.21.15.138:0/3737225084 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee840457c0 con 0x7fee940ca7b0 2024-01-21T22:52:51.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.073+0000 7fee8a000640 1 -- 172.21.15.138:0/3737225084 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fee84045920 con 0x7fee940ca7b0 2024-01-21T22:52:51.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.075+0000 7fee8a000640 1 --2- 172.21.15.138:0/3737225084 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fee6c06b330 0x7fee6c06d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:51.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.075+0000 7fee8a000640 1 -- 172.21.15.138:0/3737225084 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fee840ac890 con 0x7fee940ca7b0 2024-01-21T22:52:51.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.076+0000 7fee91400640 1 --2- 172.21.15.138:0/3737225084 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fee6c06b330 0x7fee6c06d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:51.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.077+0000 7fee91400640 1 --2- 172.21.15.138:0/3737225084 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fee6c06b330 0x7fee6c06d7f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fee94153b80 tx=0x7fee7c009060 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:51.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.084+0000 7fee8a000640 1 -- 172.21.15.138:0/3737225084 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fee840775b0 con 0x7fee940ca7b0 2024-01-21T22:52:51.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.294+0000 7fee93200640 1 -- 172.21.15.138:0/3737225084 --> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fee9406ba30 con 0x7fee6c06b330 2024-01-21T22:52:51.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.295+0000 7fee8a000640 1 -- 172.21.15.138:0/3737225084 <== mgr.4111 v2:172.21.15.152:6800/314625564 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fee7c009550 con 0x7fee6c06b330 2024-01-21T22:52:51.295 INFO:teuthology.orchestra.run.smithi138.stderr:instructed osd(s) 4 to deep-scrub 2024-01-21T22:52:51.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.297+0000 7fee93200640 1 -- 172.21.15.138:0/3737225084 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fee6c06b330 msgr2=0x7fee6c06d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:51.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.297+0000 7fee93200640 1 --2- 172.21.15.138:0/3737225084 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fee6c06b330 0x7fee6c06d7f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fee94153b80 tx=0x7fee7c009060 comp rx=0 tx=0).stop 2024-01-21T22:52:51.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.297+0000 7fee93200640 1 -- 172.21.15.138:0/3737225084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee940ca7b0 msgr2=0x7fee94151cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:51.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.297+0000 7fee93200640 1 --2- 172.21.15.138:0/3737225084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee940ca7b0 0x7fee94151cb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fee840098f0 tx=0x7fee840046c0 comp rx=0 tx=0).stop 2024-01-21T22:52:51.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.298+0000 7fee93200640 1 -- 172.21.15.138:0/3737225084 shutdown_connections 2024-01-21T22:52:51.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.298+0000 7fee93200640 1 --2- 172.21.15.138:0/3737225084 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fee6c06b330 0x7fee6c06d7f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:51.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.298+0000 7fee93200640 1 --2- 172.21.15.138:0/3737225084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee9415d1a0 0x7fee94147950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:51.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.298+0000 7fee93200640 1 --2- 172.21.15.138:0/3737225084 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee9415cdc0 0x7fee941e2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:51.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.298+0000 7fee93200640 1 --2- 172.21.15.138:0/3737225084 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee940ca7b0 0x7fee94151cb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:51.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.298+0000 7fee93200640 1 -- 172.21.15.138:0/3737225084 >> 172.21.15.138:0/3737225084 conn(0x7fee9406d370 msgr2=0x7fee94149a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:51.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.298+0000 7fee93200640 1 -- 172.21.15.138:0/3737225084 shutdown_connections 2024-01-21T22:52:51.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.298+0000 7fee93200640 1 -- 172.21.15.138:0/3737225084 wait complete. 2024-01-21T22:52:51.318 INFO:tasks.ceph:Scrubbing osd.5 2024-01-21T22:52:51.319 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-01-21T22:52:51.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.551+0000 7f892c800640 1 Processor -- start 2024-01-21T22:52:51.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.551+0000 7f892c800640 1 -- start start 2024-01-21T22:52:51.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.553+0000 7f892c800640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f8924151c00 0x7f892406d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:51.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.553+0000 7f892c800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f892406c9d0 0x7f8924072400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:51.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.553+0000 7f892c800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f892415afd0 con 0x7f8924072940 2024-01-21T22:52:51.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.553+0000 7f892c800640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f892406eb00 con 0x7f8924151c00 2024-01-21T22:52:51.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.553+0000 7f892c800640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89240c5be0 con 0x7f892406c9d0 2024-01-21T22:52:51.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.554+0000 7f8929400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f892406c9d0 0x7f8924072400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:51.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.554+0000 7f8929400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f892406c9d0 0x7f8924072400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:60166/0 (socket says 172.21.15.138:60166) 2024-01-21T22:52:51.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.554+0000 7f8929400640 1 -- 172.21.15.138:0/2318663523 learned_addr learned my addr 172.21.15.138:0/2318663523 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:51.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.556+0000 7f8929400640 1 -- 172.21.15.138:0/2318663523 >> v2:172.21.15.138:3300/0 conn(0x7f8924151c00 msgr2=0x7f892406d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:51.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.556+0000 7f8929400640 1 --2- 172.21.15.138:0/2318663523 >> v2:172.21.15.138:3300/0 conn(0x7f8924151c00 0x7f892406d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:51.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.556+0000 7f8929400640 1 -- 172.21.15.138:0/2318663523 >> v1:172.21.15.138:6789/0 conn(0x7f8924072940 legacy=0x7f892415b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:51.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.556+0000 7f8929400640 1 -- 172.21.15.138:0/2318663523 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892406ce10 con 0x7f892406c9d0 2024-01-21T22:52:51.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.558+0000 7f8929400640 1 --2- 172.21.15.138:0/2318663523 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f892406c9d0 0x7f8924072400 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f890c0098a0 tx=0x7f890c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ef33c38ea230210 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:51.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.562+0000 7f8928a00640 1 -- 172.21.15.138:0/2318663523 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890c031640 con 0x7f892406c9d0 2024-01-21T22:52:51.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.562+0000 7f8928a00640 1 -- 172.21.15.138:0/2318663523 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f890c0317a0 con 0x7f892406c9d0 2024-01-21T22:52:51.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.562+0000 7f8928a00640 1 -- 172.21.15.138:0/2318663523 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890c031b30 con 0x7f892406c9d0 2024-01-21T22:52:51.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.563+0000 7f892c800640 1 -- 172.21.15.138:0/2318663523 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f892406c9d0 msgr2=0x7f8924072400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:51.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.563+0000 7f892c800640 1 --2- 172.21.15.138:0/2318663523 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f892406c9d0 0x7f8924072400 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f890c0098a0 tx=0x7f890c02b8f0 comp rx=0 tx=0).stop 2024-01-21T22:52:51.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.563+0000 7f892c800640 1 -- 172.21.15.138:0/2318663523 shutdown_connections 2024-01-21T22:52:51.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.563+0000 7f892c800640 1 --2- 172.21.15.138:0/2318663523 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f892406c9d0 0x7f8924072400 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:51.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.563+0000 7f892c800640 1 --2- 172.21.15.138:0/2318663523 >> v2:172.21.15.138:3300/0 conn(0x7f8924151c00 0x7f892406d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:51.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.563+0000 7f892c800640 1 -- 172.21.15.138:0/2318663523 >> 172.21.15.138:0/2318663523 conn(0x7f892406add0 msgr2=0x7f89240caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:51.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.563+0000 7f892c800640 1 -- 172.21.15.138:0/2318663523 shutdown_connections 2024-01-21T22:52:51.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.563+0000 7f892c800640 1 -- 172.21.15.138:0/2318663523 wait complete. 2024-01-21T22:52:51.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.565+0000 7f892c800640 1 Processor -- start 2024-01-21T22:52:51.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.565+0000 7f892c800640 1 -- start start 2024-01-21T22:52:51.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.566+0000 7f892c800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f892406c9d0 0x7f89241f03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:51.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.566+0000 7f892c800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8924072940 0x7f8924071570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:51.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.566+0000 7f892c800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8924151c00 0x7f8924159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:51.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.566+0000 7f892c800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89240c2df0 con 0x7f8924151c00 2024-01-21T22:52:51.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.566+0000 7f892c800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89240701f0 con 0x7f8924072940 2024-01-21T22:52:51.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.566+0000 7f892c800640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89240704d0 con 0x7f892406c9d0 2024-01-21T22:52:51.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.567+0000 7f8929400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8924072940 0x7f8924071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:51.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.567+0000 7f8929e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f892406c9d0 0x7f89241f03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:51.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.567+0000 7f8929e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f892406c9d0 0x7f89241f03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:60170/0 (socket says 172.21.15.138:60170) 2024-01-21T22:52:51.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.567+0000 7f8929e00640 1 -- 172.21.15.138:0/1722781430 learned_addr learned my addr 172.21.15.138:0/1722781430 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:51.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.570+0000 7f8929e00640 1 -- 172.21.15.138:0/1722781430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8924072940 msgr2=0x7f8924071570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:51.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.570+0000 7f8929e00640 1 --2- 172.21.15.138:0/1722781430 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8924072940 0x7f8924071570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:51.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.570+0000 7f8929e00640 1 -- 172.21.15.138:0/1722781430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8924151c00 msgr2=0x7f8924159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:51.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.570+0000 7f8929e00640 1 --2- 172.21.15.138:0/1722781430 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8924151c00 0x7f8924159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:51.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.570+0000 7f8929e00640 1 -- 172.21.15.138:0/1722781430 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890c0094d0 con 0x7f892406c9d0 2024-01-21T22:52:51.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.573+0000 7f8929e00640 1 --2- 172.21.15.138:0/1722781430 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f892406c9d0 0x7f89241f03a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f891400b410 tx=0x7f891400b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:51.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.578+0000 7f8922000640 1 -- 172.21.15.138:0/1722781430 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8914011830 con 0x7f892406c9d0 2024-01-21T22:52:51.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.578+0000 7f8922000640 1 -- 172.21.15.138:0/1722781430 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89140119b0 con 0x7f892406c9d0 2024-01-21T22:52:51.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.578+0000 7f8922000640 1 -- 172.21.15.138:0/1722781430 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8914011d60 con 0x7f892406c9d0 2024-01-21T22:52:51.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.578+0000 7f892c800640 1 -- 172.21.15.138:0/1722781430 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89240707b0 con 0x7f892406c9d0 2024-01-21T22:52:51.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.578+0000 7f892c800640 1 -- 172.21.15.138:0/1722781430 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8924070d70 con 0x7f892406c9d0 2024-01-21T22:52:51.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.579+0000 7f892c800640 1 -- 172.21.15.138:0/1722781430 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f89240c2910 con 0x7f892406c9d0 2024-01-21T22:52:51.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.602+0000 7f8922000640 1 -- 172.21.15.138:0/1722781430 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f891401a400 con 0x7f892406c9d0 2024-01-21T22:52:51.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.604+0000 7f8922000640 1 --2- 172.21.15.138:0/1722781430 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f88fc06b380 0x7f88fc06d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:51.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.605+0000 7f8929400640 1 --2- 172.21.15.138:0/1722781430 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f88fc06b380 0x7f88fc06d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:51.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.605+0000 7f8922000640 1 -- 172.21.15.138:0/1722781430 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8914059310 con 0x7f892406c9d0 2024-01-21T22:52:51.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.605+0000 7f8922000640 1 --2- 172.21.15.138:0/1722781430 >> [v2:172.21.15.152:6818/4095234314,v1:172.21.15.152:6819/4095234314] conn(0x7f88fc073580 0x7f88fc0759a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:51.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.605+0000 7f8922000640 1 -- 172.21.15.138:0/1722781430 --> [v2:172.21.15.152:6818/4095234314,v1:172.21.15.152:6819/4095234314] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88fc076070 con 0x7f88fc073580 2024-01-21T22:52:51.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.605+0000 7f8922000640 1 -- 172.21.15.138:0/1722781430 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8914009230 con 0x7f892406c9d0 2024-01-21T22:52:51.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.605+0000 7f8929400640 1 --2- 172.21.15.138:0/1722781430 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f88fc06b380 0x7f88fc06d840 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f890c036000 tx=0x7f890c03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:51.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.606+0000 7f892a800640 1 --2- 172.21.15.138:0/1722781430 >> [v2:172.21.15.152:6818/4095234314,v1:172.21.15.152:6819/4095234314] conn(0x7f88fc073580 0x7f88fc0759a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:51.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.606+0000 7f892a800640 1 --2- 172.21.15.138:0/1722781430 >> [v2:172.21.15.152:6818/4095234314,v1:172.21.15.152:6819/4095234314] conn(0x7f88fc073580 0x7f88fc0759a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:51.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.607+0000 7f8922000640 1 -- 172.21.15.138:0/1722781430 <== osd.5 v2:172.21.15.152:6818/4095234314 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f88fc076070 con 0x7f88fc073580 2024-01-21T22:52:51.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.623+0000 7f892c800640 1 -- 172.21.15.138:0/1722781430 --> [v2:172.21.15.152:6818/4095234314,v1:172.21.15.152:6819/4095234314] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f89240c2630 con 0x7f88fc073580 2024-01-21T22:52:51.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.623+0000 7f8922000640 1 -- 172.21.15.138:0/1722781430 <== osd.5 v2:172.21.15.152:6818/4095234314 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f891c006b10 con 0x7f88fc073580 2024-01-21T22:52:51.624 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-01-21T22:52:51.624 INFO:teuthology.orchestra.run.smithi138.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-21T22:52:51.624 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-01-21T22:52:51.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.624+0000 7f892c800640 1 -- 172.21.15.138:0/1722781430 >> [v2:172.21.15.152:6818/4095234314,v1:172.21.15.152:6819/4095234314] conn(0x7f88fc073580 msgr2=0x7f88fc0759a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:51.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.624+0000 7f892c800640 1 --2- 172.21.15.138:0/1722781430 >> [v2:172.21.15.152:6818/4095234314,v1:172.21.15.152:6819/4095234314] conn(0x7f88fc073580 0x7f88fc0759a0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:51.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.624+0000 7f892c800640 1 -- 172.21.15.138:0/1722781430 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f88fc06b380 msgr2=0x7f88fc06d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:51.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.624+0000 7f892c800640 1 --2- 172.21.15.138:0/1722781430 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f88fc06b380 0x7f88fc06d840 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f890c036000 tx=0x7f890c03c040 comp rx=0 tx=0).stop 2024-01-21T22:52:51.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.624+0000 7f892c800640 1 -- 172.21.15.138:0/1722781430 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f892406c9d0 msgr2=0x7f89241f03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:51.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.624+0000 7f892c800640 1 --2- 172.21.15.138:0/1722781430 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f892406c9d0 0x7f89241f03a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f891400b410 tx=0x7f891400b8d0 comp rx=0 tx=0).stop 2024-01-21T22:52:51.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.624+0000 7f892a800640 1 -- 172.21.15.138:0/1722781430 reap_dead start 2024-01-21T22:52:51.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.624+0000 7f892c800640 1 -- 172.21.15.138:0/1722781430 shutdown_connections 2024-01-21T22:52:51.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.624+0000 7f892c800640 1 -- 172.21.15.138:0/1722781430 >> 172.21.15.138:0/1722781430 conn(0x7f892406add0 msgr2=0x7f892414bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:51.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.624+0000 7f892c800640 1 -- 172.21.15.138:0/1722781430 shutdown_connections 2024-01-21T22:52:51.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.624+0000 7f892c800640 1 -- 172.21.15.138:0/1722781430 wait complete. 2024-01-21T22:52:51.642 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-01-21T22:52:51.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.835+0000 7fb691400640 1 Processor -- start 2024-01-21T22:52:51.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.835+0000 7fb691400640 1 -- start start 2024-01-21T22:52:51.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.836+0000 7fb691400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb68c06c9b0 0x7fb68c0c6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:51.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.836+0000 7fb691400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fb68c0c5970 0x7fb68c0c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:51.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.836+0000 7fb691400640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb68c0c55a0 con 0x7fb68c06c5d0 2024-01-21T22:52:51.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.836+0000 7fb691400640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb68c0c3d30 con 0x7fb68c0c5970 2024-01-21T22:52:51.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.836+0000 7fb691400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c0c4310 con 0x7fb68c06c9b0 2024-01-21T22:52:51.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.837+0000 7fb68a800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7fb68c06c5d0 0x7fb68c06d110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:42480/0 (socket says 172.21.15.138:42480) 2024-01-21T22:52:51.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.837+0000 7fb68a800640 1 -- 172.21.15.138:0/545171388 learned_addr learned my addr 172.21.15.138:0/545171388 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:52:51.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.837+0000 7fb683e00640 1 --2- 172.21.15.138:0/545171388 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb68c06c9b0 0x7fb68c0c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:51.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.837+0000 7fb68b200640 1 --2- 172.21.15.138:0/545171388 >> v2:172.21.15.152:3300/0 conn(0x7fb68c0c5970 0x7fb68c0c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:51.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.840+0000 7fb68b200640 1 -- 172.21.15.138:0/545171388 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb68c06c9b0 msgr2=0x7fb68c0c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:51.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.840+0000 7fb68b200640 1 --2- 172.21.15.138:0/545171388 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb68c06c9b0 0x7fb68c0c6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:51.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.840+0000 7fb68b200640 1 -- 172.21.15.138:0/545171388 >> v1:172.21.15.138:6789/0 conn(0x7fb68c06c5d0 legacy=0x7fb68c06d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:51.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.840+0000 7fb68b200640 1 -- 172.21.15.138:0/545171388 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68c0c28d0 con 0x7fb68c0c5970 2024-01-21T22:52:51.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.842+0000 7fb68b200640 1 --2- 172.21.15.138:0/545171388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb68c0c5970 0x7fb68c0c5060 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb684009d90 tx=0x7fb68402bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf052f605ec0e486 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:51.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.846+0000 7fb689e00640 1 -- 172.21.15.138:0/545171388 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68402cce0 con 0x7fb68c0c5970 2024-01-21T22:52:51.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.846+0000 7fb689e00640 1 -- 172.21.15.138:0/545171388 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68402ce40 con 0x7fb68c0c5970 2024-01-21T22:52:51.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.846+0000 7fb689e00640 1 -- 172.21.15.138:0/545171388 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb684034660 con 0x7fb68c0c5970 2024-01-21T22:52:51.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.847+0000 7fb691400640 1 -- 172.21.15.138:0/545171388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb68c0c5970 msgr2=0x7fb68c0c5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:51.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.847+0000 7fb691400640 1 --2- 172.21.15.138:0/545171388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb68c0c5970 0x7fb68c0c5060 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb684009d90 tx=0x7fb68402bf30 comp rx=0 tx=0).stop 2024-01-21T22:52:51.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.847+0000 7fb691400640 1 -- 172.21.15.138:0/545171388 shutdown_connections 2024-01-21T22:52:51.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.847+0000 7fb691400640 1 --2- 172.21.15.138:0/545171388 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb68c06c9b0 0x7fb68c0c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:51.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.847+0000 7fb691400640 1 --2- 172.21.15.138:0/545171388 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb68c0c5970 0x7fb68c0c5060 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:51.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.847+0000 7fb691400640 1 -- 172.21.15.138:0/545171388 >> 172.21.15.138:0/545171388 conn(0x7fb68c0cb070 msgr2=0x7fb68c0cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:51.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.847+0000 7fb691400640 1 -- 172.21.15.138:0/545171388 shutdown_connections 2024-01-21T22:52:51.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.847+0000 7fb691400640 1 -- 172.21.15.138:0/545171388 wait complete. 2024-01-21T22:52:51.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.849+0000 7fb691400640 1 Processor -- start 2024-01-21T22:52:51.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.849+0000 7fb691400640 1 -- start start 2024-01-21T22:52:51.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.850+0000 7fb691400640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb68c06c5d0 0x7fb68c0afe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:51.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.850+0000 7fb691400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb68c06c9b0 0x7fb68c0ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:51.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.850+0000 7fb691400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb68c0c5970 0x7fb68c0c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:51.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.850+0000 7fb691400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c0ae5e0 con 0x7fb68c0c5970 2024-01-21T22:52:51.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.850+0000 7fb691400640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c1ef310 con 0x7fb68c06c9b0 2024-01-21T22:52:51.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.850+0000 7fb691400640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c1ef5d0 con 0x7fb68c06c5d0 2024-01-21T22:52:51.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.851+0000 7fb68a800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb68c06c5d0 0x7fb68c0afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:51.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.851+0000 7fb68a800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb68c06c5d0 0x7fb68c0afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:60188/0 (socket says 172.21.15.138:60188) 2024-01-21T22:52:51.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.851+0000 7fb68a800640 1 -- 172.21.15.138:0/3752988676 learned_addr learned my addr 172.21.15.138:0/3752988676 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:51.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.852+0000 7fb683e00640 1 --2- 172.21.15.138:0/3752988676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb68c06c9b0 0x7fb68c0ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:51.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.855+0000 7fb68a800640 1 -- 172.21.15.138:0/3752988676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb68c06c9b0 msgr2=0x7fb68c0ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:51.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.855+0000 7fb68a800640 1 --2- 172.21.15.138:0/3752988676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb68c06c9b0 0x7fb68c0ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:51.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.855+0000 7fb68a800640 1 -- 172.21.15.138:0/3752988676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb68c0c5970 msgr2=0x7fb68c0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:51.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.855+0000 7fb68a800640 1 --2- 172.21.15.138:0/3752988676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb68c0c5970 0x7fb68c0c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:51.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.855+0000 7fb68a800640 1 -- 172.21.15.138:0/3752988676 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb684009a70 con 0x7fb68c06c5d0 2024-01-21T22:52:51.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.857+0000 7fb68a800640 1 --2- 172.21.15.138:0/3752988676 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb68c06c5d0 0x7fb68c0afe80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb67c00c400 tx=0x7fb67c00c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:51.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.861+0000 7fb682a00640 1 -- 172.21.15.138:0/3752988676 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67c005ce0 con 0x7fb68c06c5d0 2024-01-21T22:52:51.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.861+0000 7fb682a00640 1 -- 172.21.15.138:0/3752988676 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67c005e40 con 0x7fb68c06c5d0 2024-01-21T22:52:51.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.861+0000 7fb682a00640 1 -- 172.21.15.138:0/3752988676 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67c0072f0 con 0x7fb68c06c5d0 2024-01-21T22:52:51.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.862+0000 7fb691400640 1 -- 172.21.15.138:0/3752988676 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb68c1ef8b0 con 0x7fb68c06c5d0 2024-01-21T22:52:51.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.862+0000 7fb691400640 1 -- 172.21.15.138:0/3752988676 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb68c1efe70 con 0x7fb68c06c5d0 2024-01-21T22:52:51.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.863+0000 7fb691400640 1 -- 172.21.15.138:0/3752988676 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb68c1f0030 con 0x7fb68c06c5d0 2024-01-21T22:52:51.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.884+0000 7fb682a00640 1 -- 172.21.15.138:0/3752988676 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fb67c007510 con 0x7fb68c06c5d0 2024-01-21T22:52:51.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.886+0000 7fb682a00640 1 --2- 172.21.15.138:0/3752988676 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fb66406b380 0x7fb66406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:51.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.886+0000 7fb682a00640 1 -- 172.21.15.138:0/3752988676 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fb67c0893a0 con 0x7fb68c06c5d0 2024-01-21T22:52:51.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.887+0000 7fb683e00640 1 --2- 172.21.15.138:0/3752988676 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fb66406b380 0x7fb66406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:51.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:51.888+0000 7fb683e00640 1 --2- 172.21.15.138:0/3752988676 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fb66406b380 0x7fb66406d840 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb68c155730 tx=0x7fb674009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:52.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.119+0000 7fb682a00640 1 -- 172.21.15.138:0/3752988676 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fb67c054130 con 0x7fb68c06c5d0 2024-01-21T22:52:52.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.321+0000 7fb691400640 1 -- 172.21.15.138:0/3752988676 --> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fb68c0c4860 con 0x7fb66406b380 2024-01-21T22:52:52.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.322+0000 7fb682a00640 1 -- 172.21.15.138:0/3752988676 <== mgr.4111 v2:172.21.15.152:6800/314625564 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb674011a30 con 0x7fb66406b380 2024-01-21T22:52:52.322 INFO:teuthology.orchestra.run.smithi138.stderr:instructed osd(s) 5 to deep-scrub 2024-01-21T22:52:52.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.324+0000 7fb691400640 1 -- 172.21.15.138:0/3752988676 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fb66406b380 msgr2=0x7fb66406d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:52.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.324+0000 7fb691400640 1 --2- 172.21.15.138:0/3752988676 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fb66406b380 0x7fb66406d840 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb68c155730 tx=0x7fb674009190 comp rx=0 tx=0).stop 2024-01-21T22:52:52.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.324+0000 7fb691400640 1 -- 172.21.15.138:0/3752988676 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb68c06c5d0 msgr2=0x7fb68c0afe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:52.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.324+0000 7fb691400640 1 --2- 172.21.15.138:0/3752988676 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb68c06c5d0 0x7fb68c0afe80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb67c00c400 tx=0x7fb67c00c8c0 comp rx=0 tx=0).stop 2024-01-21T22:52:52.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.324+0000 7fb691400640 1 -- 172.21.15.138:0/3752988676 shutdown_connections 2024-01-21T22:52:52.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.324+0000 7fb691400640 1 --2- 172.21.15.138:0/3752988676 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fb66406b380 0x7fb66406d840 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:52.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.324+0000 7fb691400640 1 --2- 172.21.15.138:0/3752988676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb68c0c5970 0x7fb68c0c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:52.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.324+0000 7fb691400640 1 --2- 172.21.15.138:0/3752988676 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fb68c06c9b0 0x7fb68c0ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:52.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.324+0000 7fb691400640 1 --2- 172.21.15.138:0/3752988676 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fb68c06c5d0 0x7fb68c0afe80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:52.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.324+0000 7fb691400640 1 -- 172.21.15.138:0/3752988676 >> 172.21.15.138:0/3752988676 conn(0x7fb68c0cb070 msgr2=0x7fb68c06a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:52.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.324+0000 7fb691400640 1 -- 172.21.15.138:0/3752988676 shutdown_connections 2024-01-21T22:52:52.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.324+0000 7fb691400640 1 -- 172.21.15.138:0/3752988676 wait complete. 2024-01-21T22:52:52.344 INFO:tasks.ceph:Scrubbing osd.6 2024-01-21T22:52:52.344 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-01-21T22:52:52.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.551+0000 7f9d3c800640 1 Processor -- start 2024-01-21T22:52:52.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.551+0000 7f9d3c800640 1 -- start start 2024-01-21T22:52:52.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.553+0000 7f9d3c800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9d340cad90 0x7f9d340c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:52.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.553+0000 7f9d3c800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d340cb1d0 0x7f9d340c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:52.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.553+0000 7f9d3c800640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d34154110 con 0x7f9d34153d30 2024-01-21T22:52:52.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.553+0000 7f9d3c800640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d340c4730 con 0x7f9d340cad90 2024-01-21T22:52:52.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.553+0000 7f9d3c800640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3406a820 con 0x7f9d340cb1d0 2024-01-21T22:52:52.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.553+0000 7f9d39400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9d340cad90 0x7f9d340c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:52.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.554+0000 7f9d3a800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d340cb1d0 0x7f9d340c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:52.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.554+0000 7f9d39400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f9d340cad90 0x7f9d340c4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:39244/0 (socket says 172.21.15.138:39244) 2024-01-21T22:52:52.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.554+0000 7f9d39400640 1 -- 172.21.15.138:0/1473017056 learned_addr learned my addr 172.21.15.138:0/1473017056 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:52.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.556+0000 7f9d3a800640 1 -- 172.21.15.138:0/1473017056 >> v2:172.21.15.152:3300/0 conn(0x7f9d340cad90 msgr2=0x7f9d340c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:52.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.556+0000 7f9d3a800640 1 --2- 172.21.15.138:0/1473017056 >> v2:172.21.15.152:3300/0 conn(0x7f9d340cad90 0x7f9d340c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:52.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.556+0000 7f9d3a800640 1 -- 172.21.15.138:0/1473017056 >> v1:172.21.15.138:6789/0 conn(0x7f9d34153d30 legacy=0x7f9d3415a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:52.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.556+0000 7f9d3a800640 1 -- 172.21.15.138:0/1473017056 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d340c4450 con 0x7f9d340cb1d0 2024-01-21T22:52:52.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.558+0000 7f9d3a800640 1 --2- 172.21.15.138:0/1473017056 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d340cb1d0 0x7f9d340c4f60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009d90 tx=0x7f9d2c02be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d3575e76ccb43af server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:52.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.562+0000 7f9d38a00640 1 -- 172.21.15.138:0/1473017056 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c02cbd0 con 0x7f9d340cb1d0 2024-01-21T22:52:52.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.562+0000 7f9d38a00640 1 -- 172.21.15.138:0/1473017056 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2c02cd30 con 0x7f9d340cb1d0 2024-01-21T22:52:52.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.562+0000 7f9d38a00640 1 -- 172.21.15.138:0/1473017056 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c02cea0 con 0x7f9d340cb1d0 2024-01-21T22:52:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.563+0000 7f9d3c800640 1 -- 172.21.15.138:0/1473017056 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d340cb1d0 msgr2=0x7f9d340c4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.563+0000 7f9d3c800640 1 --2- 172.21.15.138:0/1473017056 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d340cb1d0 0x7f9d340c4f60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009d90 tx=0x7f9d2c02be20 comp rx=0 tx=0).stop 2024-01-21T22:52:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.563+0000 7f9d3c800640 1 -- 172.21.15.138:0/1473017056 shutdown_connections 2024-01-21T22:52:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.563+0000 7f9d3c800640 1 --2- 172.21.15.138:0/1473017056 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d340cb1d0 0x7f9d340c4f60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.563+0000 7f9d3c800640 1 --2- 172.21.15.138:0/1473017056 >> v2:172.21.15.152:3300/0 conn(0x7f9d340cad90 0x7f9d340c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.563+0000 7f9d3c800640 1 -- 172.21.15.138:0/1473017056 >> 172.21.15.138:0/1473017056 conn(0x7f9d3406cee0 msgr2=0x7f9d34066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.563+0000 7f9d3c800640 1 -- 172.21.15.138:0/1473017056 shutdown_connections 2024-01-21T22:52:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.563+0000 7f9d3c800640 1 -- 172.21.15.138:0/1473017056 wait complete. 2024-01-21T22:52:52.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.566+0000 7f9d3c800640 1 Processor -- start 2024-01-21T22:52:52.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.566+0000 7f9d3c800640 1 -- start start 2024-01-21T22:52:52.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.567+0000 7f9d3c800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d340cad90 0x7f9d341f02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:52.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.567+0000 7f9d3c800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d340cb1d0 0x7f9d3415d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:52.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.567+0000 7f9d3c800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d34153d30 0x7f9d3414c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:52.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.567+0000 7f9d3c800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d341511f0 con 0x7f9d34153d30 2024-01-21T22:52:52.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.567+0000 7f9d3c800640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d34150290 con 0x7f9d340cb1d0 2024-01-21T22:52:52.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.567+0000 7f9d3c800640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d34150570 con 0x7f9d340cad90 2024-01-21T22:52:52.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.568+0000 7f9d39e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d340cad90 0x7f9d341f02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:52.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.568+0000 7f9d39400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d340cb1d0 0x7f9d3415d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:52.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.568+0000 7f9d39e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d340cad90 0x7f9d341f02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:60204/0 (socket says 172.21.15.138:60204) 2024-01-21T22:52:52.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.568+0000 7f9d39e00640 1 -- 172.21.15.138:0/1377838574 learned_addr learned my addr 172.21.15.138:0/1377838574 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:52.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.572+0000 7f9d39400640 1 -- 172.21.15.138:0/1377838574 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d340cad90 msgr2=0x7f9d341f02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:52.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.572+0000 7f9d39400640 1 --2- 172.21.15.138:0/1377838574 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d340cad90 0x7f9d341f02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:52.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.572+0000 7f9d39400640 1 -- 172.21.15.138:0/1377838574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d34153d30 msgr2=0x7f9d3414c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:52.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.572+0000 7f9d39400640 1 --2- 172.21.15.138:0/1377838574 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d34153d30 0x7f9d3414c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:52.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.572+0000 7f9d39400640 1 -- 172.21.15.138:0/1377838574 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c009a70 con 0x7f9d340cb1d0 2024-01-21T22:52:52.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.572+0000 7f9d39e00640 1 --2- 172.21.15.138:0/1377838574 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f9d340cad90 0x7f9d341f02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-21T22:52:52.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.576+0000 7f9d39400640 1 --2- 172.21.15.138:0/1377838574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d340cb1d0 0x7f9d3415d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00e870 tx=0x7f9d1c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:52.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.632+0000 7f9d32000640 1 -- 172.21.15.138:0/1377838574 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c00ee60 con 0x7f9d340cb1d0 2024-01-21T22:52:52.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.632+0000 7f9d3c800640 1 -- 172.21.15.138:0/1377838574 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d34150850 con 0x7f9d340cb1d0 2024-01-21T22:52:52.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.632+0000 7f9d3c800640 1 -- 172.21.15.138:0/1377838574 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d34150e10 con 0x7f9d340cb1d0 2024-01-21T22:52:52.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.633+0000 7f9d3c800640 1 -- 172.21.15.138:0/1377838574 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9d3406b080 con 0x7f9d340cb1d0 2024-01-21T22:52:52.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.657+0000 7f9d32000640 1 -- 172.21.15.138:0/1377838574 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1c005990 con 0x7f9d340cb1d0 2024-01-21T22:52:52.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.657+0000 7f9d32000640 1 -- 172.21.15.138:0/1377838574 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c005d00 con 0x7f9d340cb1d0 2024-01-21T22:52:52.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.659+0000 7f9d32000640 1 -- 172.21.15.138:0/1377838574 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f9d1c00c8c0 con 0x7f9d340cb1d0 2024-01-21T22:52:52.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.663+0000 7f9d32000640 1 --2- 172.21.15.138:0/1377838574 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f9d0c06b360 0x7f9d0c06d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:52.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.664+0000 7f9d32000640 1 -- 172.21.15.138:0/1377838574 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f9d1c08a2a0 con 0x7f9d340cb1d0 2024-01-21T22:52:52.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.664+0000 7f9d39e00640 1 --2- 172.21.15.138:0/1377838574 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f9d0c06b360 0x7f9d0c06d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:52.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.665+0000 7f9d32000640 1 --2- 172.21.15.138:0/1377838574 >> [v2:172.21.15.152:6802/223848127,v1:172.21.15.152:6803/223848127] conn(0x7f9d0c073560 0x7f9d0c075980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:52.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.665+0000 7f9d39e00640 1 --2- 172.21.15.138:0/1377838574 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f9d0c06b360 0x7f9d0c06d820 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9d24001c80 tx=0x7f9d24009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:52.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.665+0000 7f9d32000640 1 -- 172.21.15.138:0/1377838574 --> [v2:172.21.15.152:6802/223848127,v1:172.21.15.152:6803/223848127] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9d0c076050 con 0x7f9d0c073560 2024-01-21T22:52:52.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.665+0000 7f9d3a800640 1 --2- 172.21.15.138:0/1377838574 >> [v2:172.21.15.152:6802/223848127,v1:172.21.15.152:6803/223848127] conn(0x7f9d0c073560 0x7f9d0c075980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:52.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.666+0000 7f9d3a800640 1 --2- 172.21.15.138:0/1377838574 >> [v2:172.21.15.152:6802/223848127,v1:172.21.15.152:6803/223848127] conn(0x7f9d0c073560 0x7f9d0c075980 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:52.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.667+0000 7f9d32000640 1 -- 172.21.15.138:0/1377838574 <== osd.6 v2:172.21.15.152:6802/223848127 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9d0c076050 con 0x7f9d0c073560 2024-01-21T22:52:52.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.686+0000 7f9d3c800640 1 -- 172.21.15.138:0/1377838574 --> [v2:172.21.15.152:6802/223848127,v1:172.21.15.152:6803/223848127] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9d3406ada0 con 0x7f9d0c073560 2024-01-21T22:52:52.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.687+0000 7f9d32000640 1 -- 172.21.15.138:0/1377838574 <== osd.6 v2:172.21.15.152:6802/223848127 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9d2c03f450 con 0x7f9d0c073560 2024-01-21T22:52:52.687 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-01-21T22:52:52.687 INFO:teuthology.orchestra.run.smithi138.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-21T22:52:52.688 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-01-21T22:52:52.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.687+0000 7f9d3c800640 1 -- 172.21.15.138:0/1377838574 >> [v2:172.21.15.152:6802/223848127,v1:172.21.15.152:6803/223848127] conn(0x7f9d0c073560 msgr2=0x7f9d0c075980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:52.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.687+0000 7f9d3c800640 1 --2- 172.21.15.138:0/1377838574 >> [v2:172.21.15.152:6802/223848127,v1:172.21.15.152:6803/223848127] conn(0x7f9d0c073560 0x7f9d0c075980 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:52.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.688+0000 7f9d3c800640 1 -- 172.21.15.138:0/1377838574 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f9d0c06b360 msgr2=0x7f9d0c06d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:52.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.688+0000 7f9d3c800640 1 --2- 172.21.15.138:0/1377838574 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f9d0c06b360 0x7f9d0c06d820 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9d24001c80 tx=0x7f9d24009040 comp rx=0 tx=0).stop 2024-01-21T22:52:52.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.688+0000 7f9d3c800640 1 -- 172.21.15.138:0/1377838574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d340cb1d0 msgr2=0x7f9d3415d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:52.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.688+0000 7f9d3c800640 1 --2- 172.21.15.138:0/1377838574 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f9d340cb1d0 0x7f9d3415d190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c00e870 tx=0x7f9d1c00ed30 comp rx=0 tx=0).stop 2024-01-21T22:52:52.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.688+0000 7f9d3a800640 1 -- 172.21.15.138:0/1377838574 reap_dead start 2024-01-21T22:52:52.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.688+0000 7f9d3c800640 1 -- 172.21.15.138:0/1377838574 shutdown_connections 2024-01-21T22:52:52.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.688+0000 7f9d3c800640 1 -- 172.21.15.138:0/1377838574 >> 172.21.15.138:0/1377838574 conn(0x7f9d3406cee0 msgr2=0x7f9d340c4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:52.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.688+0000 7f9d3c800640 1 -- 172.21.15.138:0/1377838574 shutdown_connections 2024-01-21T22:52:52.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.688+0000 7f9d3c800640 1 -- 172.21.15.138:0/1377838574 wait complete. 2024-01-21T22:52:52.714 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-01-21T22:52:52.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.926+0000 7fe36f200640 1 Processor -- start 2024-01-21T22:52:52.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.926+0000 7fe36f200640 1 -- start start 2024-01-21T22:52:52.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.928+0000 7fe36f200640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe37015cdc0 0x7fe3700c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:52.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.928+0000 7fe36f200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe37015d1a0 0x7fe3700c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:52.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.928+0000 7fe36f200640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3700c4900 con 0x7fe3700ca7b0 2024-01-21T22:52:52.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.928+0000 7fe36f200640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3700c52d0 con 0x7fe37015cdc0 2024-01-21T22:52:52.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.928+0000 7fe36f200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe37015dc90 con 0x7fe37015d1a0 2024-01-21T22:52:52.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.929+0000 7fe36e800640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe37015d1a0 0x7fe3700c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:52.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.929+0000 7fe36de00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fe37015cdc0 0x7fe3700c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:52.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.929+0000 7fe36d400640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fe3700ca7b0 0x7fe37006c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:34288/0 (socket says 172.21.15.138:34288) 2024-01-21T22:52:52.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.929+0000 7fe36d400640 1 -- 172.21.15.138:0/3226718880 learned_addr learned my addr 172.21.15.138:0/3226718880 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:52:52.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.932+0000 7fe36e800640 1 -- 172.21.15.138:0/3226718880 >> v2:172.21.15.138:3300/0 conn(0x7fe37015cdc0 msgr2=0x7fe3700c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:52.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.932+0000 7fe36e800640 1 --2- 172.21.15.138:0/3226718880 >> v2:172.21.15.138:3300/0 conn(0x7fe37015cdc0 0x7fe3700c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:52.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.932+0000 7fe36e800640 1 -- 172.21.15.138:0/3226718880 >> v1:172.21.15.152:6789/0 conn(0x7fe3700ca7b0 legacy=0x7fe37006c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:52.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.932+0000 7fe36e800640 1 -- 172.21.15.138:0/3226718880 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37006b4b0 con 0x7fe37015d1a0 2024-01-21T22:52:52.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.935+0000 7fe36e800640 1 --2- 172.21.15.138:0/3226718880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe37015d1a0 0x7fe3700c4d90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe370074a90 tx=0x7fe36802b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87136af07b58511e server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:52.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.939+0000 7fe36ca00640 1 -- 172.21.15.138:0/3226718880 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368030610 con 0x7fe37015d1a0 2024-01-21T22:52:52.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.939+0000 7fe36ca00640 1 -- 172.21.15.138:0/3226718880 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe368030770 con 0x7fe37015d1a0 2024-01-21T22:52:52.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.939+0000 7fe36ca00640 1 -- 172.21.15.138:0/3226718880 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368030b00 con 0x7fe37015d1a0 2024-01-21T22:52:52.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.940+0000 7fe36f200640 1 -- 172.21.15.138:0/3226718880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe37015d1a0 msgr2=0x7fe3700c4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:52.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.940+0000 7fe36f200640 1 --2- 172.21.15.138:0/3226718880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe37015d1a0 0x7fe3700c4d90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe370074a90 tx=0x7fe36802b960 comp rx=0 tx=0).stop 2024-01-21T22:52:52.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.940+0000 7fe36f200640 1 -- 172.21.15.138:0/3226718880 shutdown_connections 2024-01-21T22:52:52.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.940+0000 7fe36f200640 1 --2- 172.21.15.138:0/3226718880 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe37015d1a0 0x7fe3700c4d90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:52.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.940+0000 7fe36f200640 1 --2- 172.21.15.138:0/3226718880 >> v2:172.21.15.138:3300/0 conn(0x7fe37015cdc0 0x7fe3700c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:52.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.940+0000 7fe36f200640 1 -- 172.21.15.138:0/3226718880 >> 172.21.15.138:0/3226718880 conn(0x7fe37006d370 msgr2=0x7fe370066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:52.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.940+0000 7fe36f200640 1 -- 172.21.15.138:0/3226718880 shutdown_connections 2024-01-21T22:52:52.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.940+0000 7fe36f200640 1 -- 172.21.15.138:0/3226718880 wait complete. 2024-01-21T22:52:52.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.942+0000 7fe36f200640 1 Processor -- start 2024-01-21T22:52:52.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.942+0000 7fe36f200640 1 -- start start 2024-01-21T22:52:52.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.943+0000 7fe36f200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3700ca7b0 0x7fe3700b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:52.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.943+0000 7fe36f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe37015cdc0 0x7fe37015a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:52.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.943+0000 7fe36f200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe37015d1a0 0x7fe37015c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:52.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.943+0000 7fe36f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3700c67c0 con 0x7fe37015cdc0 2024-01-21T22:52:52.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.943+0000 7fe36f200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37015c9f0 con 0x7fe37015d1a0 2024-01-21T22:52:52.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.943+0000 7fe36f200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3701513b0 con 0x7fe3700ca7b0 2024-01-21T22:52:52.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.944+0000 7fe36e800640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe37015d1a0 0x7fe37015c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:52.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.944+0000 7fe36d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe37015cdc0 0x7fe37015a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:52.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.944+0000 7fe36de00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3700ca7b0 0x7fe3700b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:52.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.944+0000 7fe36d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe37015cdc0 0x7fe37015a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60770/0 (socket says 172.21.15.138:60770) 2024-01-21T22:52:52.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.944+0000 7fe36d400640 1 -- 172.21.15.138:0/2457896921 learned_addr learned my addr 172.21.15.138:0/2457896921 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:52.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.947+0000 7fe36e800640 1 -- 172.21.15.138:0/2457896921 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3700ca7b0 msgr2=0x7fe3700b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:52.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.947+0000 7fe36e800640 1 --2- 172.21.15.138:0/2457896921 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3700ca7b0 0x7fe3700b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:52.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.947+0000 7fe36e800640 1 -- 172.21.15.138:0/2457896921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe37015cdc0 msgr2=0x7fe37015a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:52.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.947+0000 7fe36e800640 1 --2- 172.21.15.138:0/2457896921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe37015cdc0 0x7fe37015a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:52.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.947+0000 7fe36e800640 1 -- 172.21.15.138:0/2457896921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe368009400 con 0x7fe37015d1a0 2024-01-21T22:52:52.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.950+0000 7fe36e800640 1 --2- 172.21.15.138:0/2457896921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe37015d1a0 0x7fe37015c4b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe368033000 tx=0x7fe3680046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:52.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.953+0000 7fe366000640 1 -- 172.21.15.138:0/2457896921 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36803e070 con 0x7fe37015d1a0 2024-01-21T22:52:52.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.953+0000 7fe366000640 1 -- 172.21.15.138:0/2457896921 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe368039910 con 0x7fe37015d1a0 2024-01-21T22:52:52.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.953+0000 7fe366000640 1 -- 172.21.15.138:0/2457896921 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe368039ca0 con 0x7fe37015d1a0 2024-01-21T22:52:52.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.953+0000 7fe36f200640 1 -- 172.21.15.138:0/2457896921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe370151690 con 0x7fe37015d1a0 2024-01-21T22:52:52.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.954+0000 7fe36f200640 1 -- 172.21.15.138:0/2457896921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe370152b10 con 0x7fe37015d1a0 2024-01-21T22:52:52.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:52.955+0000 7fe36f200640 1 -- 172.21.15.138:0/2457896921 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe370074ee0 con 0x7fe37015d1a0 2024-01-21T22:52:53.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.206+0000 7fe366000640 1 -- 172.21.15.138:0/2457896921 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fe36804b400 con 0x7fe37015d1a0 2024-01-21T22:52:53.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.209+0000 7fe366000640 1 --2- 172.21.15.138:0/2457896921 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe34806b380 0x7fe34806d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:53.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.209+0000 7fe366000640 1 -- 172.21.15.138:0/2457896921 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fe3680ac910 con 0x7fe37015d1a0 2024-01-21T22:52:53.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.209+0000 7fe36de00640 1 --2- 172.21.15.138:0/2457896921 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe34806b380 0x7fe34806d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:53.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.211+0000 7fe36de00640 1 --2- 172.21.15.138:0/2457896921 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe34806b380 0x7fe34806d840 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe358009b40 tx=0x7fe358009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:53.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.216+0000 7fe366000640 1 -- 172.21.15.138:0/2457896921 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe3680776a0 con 0x7fe37015d1a0 2024-01-21T22:52:53.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.423+0000 7fe36f200640 1 -- 172.21.15.138:0/2457896921 --> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fe37006ba30 con 0x7fe34806b380 2024-01-21T22:52:53.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.424+0000 7fe366000640 1 -- 172.21.15.138:0/2457896921 <== mgr.4111 v2:172.21.15.152:6800/314625564 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe358011a60 con 0x7fe34806b380 2024-01-21T22:52:53.424 INFO:teuthology.orchestra.run.smithi138.stderr:instructed osd(s) 6 to deep-scrub 2024-01-21T22:52:53.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.426+0000 7fe36f200640 1 -- 172.21.15.138:0/2457896921 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe34806b380 msgr2=0x7fe34806d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:53.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.426+0000 7fe36f200640 1 --2- 172.21.15.138:0/2457896921 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe34806b380 0x7fe34806d840 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe358009b40 tx=0x7fe358009190 comp rx=0 tx=0).stop 2024-01-21T22:52:53.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.426+0000 7fe36f200640 1 -- 172.21.15.138:0/2457896921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe37015d1a0 msgr2=0x7fe37015c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:53.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.426+0000 7fe36f200640 1 --2- 172.21.15.138:0/2457896921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe37015d1a0 0x7fe37015c4b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe368033000 tx=0x7fe3680046c0 comp rx=0 tx=0).stop 2024-01-21T22:52:53.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.427+0000 7fe36f200640 1 -- 172.21.15.138:0/2457896921 shutdown_connections 2024-01-21T22:52:53.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.427+0000 7fe36f200640 1 --2- 172.21.15.138:0/2457896921 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe34806b380 0x7fe34806d840 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:53.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.427+0000 7fe36f200640 1 --2- 172.21.15.138:0/2457896921 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe37015d1a0 0x7fe37015c4b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:53.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.427+0000 7fe36f200640 1 --2- 172.21.15.138:0/2457896921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe37015cdc0 0x7fe37015a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:53.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.427+0000 7fe36f200640 1 --2- 172.21.15.138:0/2457896921 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3700ca7b0 0x7fe3700b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:53.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.427+0000 7fe36f200640 1 -- 172.21.15.138:0/2457896921 >> 172.21.15.138:0/2457896921 conn(0x7fe37006d370 msgr2=0x7fe370066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:53.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.427+0000 7fe36f200640 1 -- 172.21.15.138:0/2457896921 shutdown_connections 2024-01-21T22:52:53.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.427+0000 7fe36f200640 1 -- 172.21.15.138:0/2457896921 wait complete. 2024-01-21T22:52:53.446 INFO:tasks.ceph:Scrubbing osd.7 2024-01-21T22:52:53.447 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-01-21T22:52:53.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.652+0000 7fee7de00640 1 Processor -- start 2024-01-21T22:52:53.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.652+0000 7fee7de00640 1 -- start start 2024-01-21T22:52:53.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.654+0000 7fee7de00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fee7806c050 0x7fee781568b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:53.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.654+0000 7fee7de00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fee78070d30 0x7fee7806d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:53.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.654+0000 7fee7de00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee7806d690 con 0x7fee7806c430 2024-01-21T22:52:53.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.654+0000 7fee7de00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee780c5930 con 0x7fee7806c050 2024-01-21T22:52:53.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.654+0000 7fee7de00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee78156ef0 con 0x7fee78070d30 2024-01-21T22:52:53.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.655+0000 7fee6fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fee78070d30 0x7fee7806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:53.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.655+0000 7fee6fe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fee78070d30 0x7fee7806d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:39290/0 (socket says 172.21.15.138:39290) 2024-01-21T22:52:53.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.655+0000 7fee6fe00640 1 -- 172.21.15.138:0/3510459548 learned_addr learned my addr 172.21.15.138:0/3510459548 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:53.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.656+0000 7fee76a00640 1 --2- 172.21.15.138:0/3510459548 >> v2:172.21.15.138:3300/0 conn(0x7fee7806c050 0x7fee781568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:53.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.657+0000 7fee6fe00640 1 -- 172.21.15.138:0/3510459548 >> v2:172.21.15.138:3300/0 conn(0x7fee7806c050 msgr2=0x7fee781568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:53.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.657+0000 7fee6fe00640 1 --2- 172.21.15.138:0/3510459548 >> v2:172.21.15.138:3300/0 conn(0x7fee7806c050 0x7fee781568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:53.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.657+0000 7fee6fe00640 1 -- 172.21.15.138:0/3510459548 >> v1:172.21.15.138:6789/0 conn(0x7fee7806c430 legacy=0x7fee78150770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:53.658 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.657+0000 7fee6fe00640 1 -- 172.21.15.138:0/3510459548 --> v2:172.21.15.152:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee781571d0 con 0x7fee78070d30 2024-01-21T22:52:53.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.659+0000 7fee6fe00640 1 --2- 172.21.15.138:0/3510459548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee78070d30 0x7fee7806d150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fee780cb9c0 tx=0x7fee6002b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecdfd623034b52a server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:53.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.667+0000 7fee76000640 1 -- 172.21.15.138:0/3510459548 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee60030860 con 0x7fee78070d30 2024-01-21T22:52:53.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.667+0000 7fee76000640 1 -- 172.21.15.138:0/3510459548 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee600309c0 con 0x7fee78070d30 2024-01-21T22:52:53.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.667+0000 7fee76000640 1 -- 172.21.15.138:0/3510459548 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee60030d50 con 0x7fee78070d30 2024-01-21T22:52:53.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.668+0000 7fee7de00640 1 -- 172.21.15.138:0/3510459548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee78070d30 msgr2=0x7fee7806d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:53.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.668+0000 7fee7de00640 1 --2- 172.21.15.138:0/3510459548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee78070d30 0x7fee7806d150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fee780cb9c0 tx=0x7fee6002b9a0 comp rx=0 tx=0).stop 2024-01-21T22:52:53.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.668+0000 7fee7de00640 1 -- 172.21.15.138:0/3510459548 shutdown_connections 2024-01-21T22:52:53.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.668+0000 7fee7de00640 1 --2- 172.21.15.138:0/3510459548 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee78070d30 0x7fee7806d150 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:53.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.668+0000 7fee7de00640 1 --2- 172.21.15.138:0/3510459548 >> v2:172.21.15.138:3300/0 conn(0x7fee7806c050 0x7fee781568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:53.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.668+0000 7fee7de00640 1 -- 172.21.15.138:0/3510459548 >> 172.21.15.138:0/3510459548 conn(0x7fee7806add0 msgr2=0x7fee780caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:53.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.668+0000 7fee7de00640 1 -- 172.21.15.138:0/3510459548 shutdown_connections 2024-01-21T22:52:53.669 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.668+0000 7fee7de00640 1 -- 172.21.15.138:0/3510459548 wait complete. 2024-01-21T22:52:53.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.670+0000 7fee7de00640 1 Processor -- start 2024-01-21T22:52:53.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.671+0000 7fee7de00640 1 -- start start 2024-01-21T22:52:53.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.672+0000 7fee7de00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee7806c050 0x7fee780bfae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:53.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.672+0000 7fee7de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee7806c430 0x7fee781494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:53.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.672+0000 7fee7de00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee78070d30 0x7fee78149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:53.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.672+0000 7fee7de00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee780c0020 con 0x7fee7806c430 2024-01-21T22:52:53.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.672+0000 7fee7de00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7814b380 con 0x7fee78070d30 2024-01-21T22:52:53.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.672+0000 7fee7de00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7814b0a0 con 0x7fee7806c050 2024-01-21T22:52:53.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.673+0000 7fee76a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee7806c050 0x7fee780bfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:53.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.673+0000 7fee77400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee78070d30 0x7fee78149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:53.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.673+0000 7fee77400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee78070d30 0x7fee78149a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:39306/0 (socket says 172.21.15.138:39306) 2024-01-21T22:52:53.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.673+0000 7fee77400640 1 -- 172.21.15.138:0/230768339 learned_addr learned my addr 172.21.15.138:0/230768339 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:53.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.676+0000 7fee77400640 1 -- 172.21.15.138:0/230768339 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee7806c050 msgr2=0x7fee780bfae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:53.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.676+0000 7fee77400640 1 --2- 172.21.15.138:0/230768339 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fee7806c050 0x7fee780bfae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:53.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.677+0000 7fee77400640 1 -- 172.21.15.138:0/230768339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee7806c430 msgr2=0x7fee781494e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:53.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.677+0000 7fee77400640 1 --2- 172.21.15.138:0/230768339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee7806c430 0x7fee781494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:53.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.677+0000 7fee77400640 1 -- 172.21.15.138:0/230768339 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee600094d0 con 0x7fee78070d30 2024-01-21T22:52:53.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.679+0000 7fee77400640 1 --2- 172.21.15.138:0/230768339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee78070d30 0x7fee78149a20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fee7000c780 tx=0x7fee7000cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:53.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.683+0000 7fee6f400640 1 -- 172.21.15.138:0/230768339 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee70003d80 con 0x7fee78070d30 2024-01-21T22:52:53.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.683+0000 7fee6f400640 1 -- 172.21.15.138:0/230768339 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7000ce30 con 0x7fee78070d30 2024-01-21T22:52:53.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.683+0000 7fee6f400640 1 -- 172.21.15.138:0/230768339 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee700052b0 con 0x7fee78070d30 2024-01-21T22:52:53.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.683+0000 7fee7de00640 1 -- 172.21.15.138:0/230768339 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee780be9f0 con 0x7fee78070d30 2024-01-21T22:52:53.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.683+0000 7fee7de00640 1 -- 172.21.15.138:0/230768339 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee780befb0 con 0x7fee78070d30 2024-01-21T22:52:53.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.684+0000 7fee7de00640 1 -- 172.21.15.138:0/230768339 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fee78157790 con 0x7fee78070d30 2024-01-21T22:52:53.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.771+0000 7fee6f400640 1 -- 172.21.15.138:0/230768339 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fee70005410 con 0x7fee78070d30 2024-01-21T22:52:53.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.773+0000 7fee6f400640 1 --2- 172.21.15.138:0/230768339 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fee5006b3b0 0x7fee5006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:53.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.774+0000 7fee6f400640 1 -- 172.21.15.138:0/230768339 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fee70089940 con 0x7fee78070d30 2024-01-21T22:52:53.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.774+0000 7fee76a00640 1 --2- 172.21.15.138:0/230768339 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fee5006b3b0 0x7fee5006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:53.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.774+0000 7fee6f400640 1 --2- 172.21.15.138:0/230768339 >> [v2:172.21.15.152:6810/52509095,v1:172.21.15.152:6811/52509095] conn(0x7fee500735b0 0x7fee500759d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:53.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.774+0000 7fee6f400640 1 -- 172.21.15.138:0/230768339 --> [v2:172.21.15.152:6810/52509095,v1:172.21.15.152:6811/52509095] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee500760a0 con 0x7fee500735b0 2024-01-21T22:52:53.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.775+0000 7fee6fe00640 1 --2- 172.21.15.138:0/230768339 >> [v2:172.21.15.152:6810/52509095,v1:172.21.15.152:6811/52509095] conn(0x7fee500735b0 0x7fee500759d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:53.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.775+0000 7fee76a00640 1 --2- 172.21.15.138:0/230768339 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fee5006b3b0 0x7fee5006d870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fee6800aac0 tx=0x7fee68008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:53.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.776+0000 7fee6fe00640 1 --2- 172.21.15.138:0/230768339 >> [v2:172.21.15.152:6810/52509095,v1:172.21.15.152:6811/52509095] conn(0x7fee500735b0 0x7fee500759d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:53.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.777+0000 7fee6f400640 1 -- 172.21.15.138:0/230768339 <== osd.7 v2:172.21.15.152:6810/52509095 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fee500760a0 con 0x7fee500735b0 2024-01-21T22:52:53.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.779+0000 7fee6f400640 1 -- 172.21.15.138:0/230768339 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee70054740 con 0x7fee78070d30 2024-01-21T22:52:53.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.795+0000 7fee7de00640 1 -- 172.21.15.138:0/230768339 --> [v2:172.21.15.152:6810/52509095,v1:172.21.15.152:6811/52509095] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fee780c5bd0 con 0x7fee500735b0 2024-01-21T22:52:53.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.795+0000 7fee6f400640 1 -- 172.21.15.138:0/230768339 <== osd.7 v2:172.21.15.152:6810/52509095 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fee60030b30 con 0x7fee500735b0 2024-01-21T22:52:53.796 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-01-21T22:52:53.796 INFO:teuthology.orchestra.run.smithi138.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-01-21T22:52:53.796 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-01-21T22:52:53.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.796+0000 7fee7de00640 1 -- 172.21.15.138:0/230768339 >> [v2:172.21.15.152:6810/52509095,v1:172.21.15.152:6811/52509095] conn(0x7fee500735b0 msgr2=0x7fee500759d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:53.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.796+0000 7fee7de00640 1 --2- 172.21.15.138:0/230768339 >> [v2:172.21.15.152:6810/52509095,v1:172.21.15.152:6811/52509095] conn(0x7fee500735b0 0x7fee500759d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:53.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.796+0000 7fee7de00640 1 -- 172.21.15.138:0/230768339 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fee5006b3b0 msgr2=0x7fee5006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:53.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.796+0000 7fee7de00640 1 --2- 172.21.15.138:0/230768339 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fee5006b3b0 0x7fee5006d870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fee6800aac0 tx=0x7fee68008040 comp rx=0 tx=0).stop 2024-01-21T22:52:53.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.796+0000 7fee7de00640 1 -- 172.21.15.138:0/230768339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee78070d30 msgr2=0x7fee78149a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:53.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.796+0000 7fee7de00640 1 --2- 172.21.15.138:0/230768339 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fee78070d30 0x7fee78149a20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fee7000c780 tx=0x7fee7000cc40 comp rx=0 tx=0).stop 2024-01-21T22:52:53.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.796+0000 7fee77400640 1 -- 172.21.15.138:0/230768339 reap_dead start 2024-01-21T22:52:53.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.796+0000 7fee7de00640 1 -- 172.21.15.138:0/230768339 shutdown_connections 2024-01-21T22:52:53.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.796+0000 7fee7de00640 1 -- 172.21.15.138:0/230768339 >> 172.21.15.138:0/230768339 conn(0x7fee7806add0 msgr2=0x7fee780c60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:53.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.796+0000 7fee7de00640 1 -- 172.21.15.138:0/230768339 shutdown_connections 2024-01-21T22:52:53.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:53.796+0000 7fee7de00640 1 -- 172.21.15.138:0/230768339 wait complete. 2024-01-21T22:52:53.821 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-01-21T22:52:54.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.024+0000 7f1b51000640 1 Processor -- start 2024-01-21T22:52:54.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.024+0000 7f1b51000640 1 -- start start 2024-01-21T22:52:54.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.025+0000 7f1b51000640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1b4c0ca9e0 0x7f1b4c1577c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:54.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.025+0000 7f1b51000640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1b4c066b60 0x7f1b4c06e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:54.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.025+0000 7f1b51000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b4c0c7be0 0x7f1b4c06ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:54.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.025+0000 7f1b51000640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b4c0cdbf0 con 0x7f1b4c0ca9e0 2024-01-21T22:52:54.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.025+0000 7f1b51000640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b4c06f620 con 0x7f1b4c066b60 2024-01-21T22:52:54.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.025+0000 7f1b51000640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c155250 con 0x7f1b4c0c7be0 2024-01-21T22:52:54.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.026+0000 7f1b4aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1b4c066b60 0x7f1b4c06e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:54.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.026+0000 7f1b4be00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b4c0c7be0 0x7f1b4c06ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:54.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.026+0000 7f1b4b400640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f1b4c0ca9e0 0x7f1b4c1577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:54.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.026+0000 7f1b4aa00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f1b4c066b60 0x7f1b4c06e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:39308/0 (socket says 172.21.15.138:39308) 2024-01-21T22:52:54.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.026+0000 7f1b4aa00640 1 -- 172.21.15.138:0/2227416576 learned_addr learned my addr 172.21.15.138:0/2227416576 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:54.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.029+0000 7f1b4be00640 1 -- 172.21.15.138:0/2227416576 >> v2:172.21.15.152:3300/0 conn(0x7f1b4c066b60 msgr2=0x7f1b4c06e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:54.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.029+0000 7f1b4be00640 1 --2- 172.21.15.138:0/2227416576 >> v2:172.21.15.152:3300/0 conn(0x7f1b4c066b60 0x7f1b4c06e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:54.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.029+0000 7f1b4be00640 1 -- 172.21.15.138:0/2227416576 >> v2:172.21.15.138:3300/0 conn(0x7f1b4c0ca9e0 msgr2=0x7f1b4c1577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:54.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.029+0000 7f1b4be00640 1 --2- 172.21.15.138:0/2227416576 >> v2:172.21.15.138:3300/0 conn(0x7f1b4c0ca9e0 0x7f1b4c1577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:54.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.029+0000 7f1b4be00640 1 -- 172.21.15.138:0/2227416576 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4c0cd3b0 con 0x7f1b4c0c7be0 2024-01-21T22:52:54.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.031+0000 7f1b4be00640 1 --2- 172.21.15.138:0/2227416576 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b4c0c7be0 0x7f1b4c06ec40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0cdeb0 tx=0x7f1b4402bec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f4dce4efd11ca40 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:54.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.034+0000 7f1b4a000640 1 -- 172.21.15.138:0/2227416576 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4402cc70 con 0x7f1b4c0c7be0 2024-01-21T22:52:54.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.034+0000 7f1b4a000640 1 -- 172.21.15.138:0/2227416576 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4402cdd0 con 0x7f1b4c0c7be0 2024-01-21T22:52:54.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.034+0000 7f1b4a000640 1 -- 172.21.15.138:0/2227416576 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44034660 con 0x7f1b4c0c7be0 2024-01-21T22:52:54.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.035+0000 7f1b51000640 1 -- 172.21.15.138:0/2227416576 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b4c0c7be0 msgr2=0x7f1b4c06ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:54.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.035+0000 7f1b51000640 1 --2- 172.21.15.138:0/2227416576 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b4c0c7be0 0x7f1b4c06ec40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0cdeb0 tx=0x7f1b4402bec0 comp rx=0 tx=0).stop 2024-01-21T22:52:54.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.035+0000 7f1b51000640 1 -- 172.21.15.138:0/2227416576 shutdown_connections 2024-01-21T22:52:54.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.035+0000 7f1b51000640 1 --2- 172.21.15.138:0/2227416576 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b4c0c7be0 0x7f1b4c06ec40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:54.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.035+0000 7f1b51000640 1 --2- 172.21.15.138:0/2227416576 >> v2:172.21.15.152:3300/0 conn(0x7f1b4c066b60 0x7f1b4c06e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:54.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.035+0000 7f1b51000640 1 --2- 172.21.15.138:0/2227416576 >> v2:172.21.15.138:3300/0 conn(0x7f1b4c0ca9e0 0x7f1b4c1577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:54.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.035+0000 7f1b51000640 1 -- 172.21.15.138:0/2227416576 >> 172.21.15.138:0/2227416576 conn(0x7f1b4c06a8d0 msgr2=0x7f1b4c066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:54.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.036+0000 7f1b51000640 1 -- 172.21.15.138:0/2227416576 shutdown_connections 2024-01-21T22:52:54.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.036+0000 7f1b51000640 1 -- 172.21.15.138:0/2227416576 wait complete. 2024-01-21T22:52:54.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.038+0000 7f1b51000640 1 Processor -- start 2024-01-21T22:52:54.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.038+0000 7f1b51000640 1 -- start start 2024-01-21T22:52:54.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.039+0000 7f1b51000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b4c066b60 0x7f1b4c0c21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:54.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.039+0000 7f1b51000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b4c0c7be0 0x7f1b4c0c2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:54.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.039+0000 7f1b51000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b4c0ca9e0 0x7f1b4c148770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:54.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.039+0000 7f1b51000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c0c1db0 con 0x7f1b4c066b60 2024-01-21T22:52:54.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.039+0000 7f1b51000640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c1572f0 con 0x7f1b4c0ca9e0 2024-01-21T22:52:54.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.039+0000 7f1b51000640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c0c11c0 con 0x7f1b4c0c7be0 2024-01-21T22:52:54.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.039+0000 7f1b4aa00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b4c0c7be0 0x7f1b4c0c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:54.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.039+0000 7f1b4b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b4c066b60 0x7f1b4c0c21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:54.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.039+0000 7f1b4be00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b4c0ca9e0 0x7f1b4c148770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:54.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.040+0000 7f1b4aa00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b4c0c7be0 0x7f1b4c0c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:60250/0 (socket says 172.21.15.138:60250) 2024-01-21T22:52:54.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.040+0000 7f1b4b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b4c066b60 0x7f1b4c0c21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60800/0 (socket says 172.21.15.138:60800) 2024-01-21T22:52:54.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.040+0000 7f1b4aa00640 1 -- 172.21.15.138:0/2275961306 learned_addr learned my addr 172.21.15.138:0/2275961306 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:54.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.043+0000 7f1b4be00640 1 -- 172.21.15.138:0/2275961306 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b4c0c7be0 msgr2=0x7f1b4c0c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:54.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.043+0000 7f1b4be00640 1 --2- 172.21.15.138:0/2275961306 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b4c0c7be0 0x7f1b4c0c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:54.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.043+0000 7f1b4be00640 1 -- 172.21.15.138:0/2275961306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b4c066b60 msgr2=0x7f1b4c0c21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:54.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.043+0000 7f1b4be00640 1 --2- 172.21.15.138:0/2275961306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b4c066b60 0x7f1b4c0c21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:54.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.043+0000 7f1b4be00640 1 -- 172.21.15.138:0/2275961306 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b44009a70 con 0x7f1b4c0ca9e0 2024-01-21T22:52:54.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.043+0000 7f1b4aa00640 1 --2- 172.21.15.138:0/2275961306 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b4c0c7be0 0x7f1b4c0c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-21T22:52:54.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.045+0000 7f1b4be00640 1 --2- 172.21.15.138:0/2275961306 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b4c0ca9e0 0x7f1b4c148770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c155390 tx=0x7f1b44004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:54.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.049+0000 7f1b43400640 1 -- 172.21.15.138:0/2275961306 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4403e610 con 0x7f1b4c0ca9e0 2024-01-21T22:52:54.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.049+0000 7f1b43400640 1 -- 172.21.15.138:0/2275961306 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4403e770 con 0x7f1b4c0ca9e0 2024-01-21T22:52:54.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.049+0000 7f1b43400640 1 -- 172.21.15.138:0/2275961306 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4403eb20 con 0x7f1b4c0ca9e0 2024-01-21T22:52:54.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.049+0000 7f1b51000640 1 -- 172.21.15.138:0/2275961306 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4c157010 con 0x7f1b4c0ca9e0 2024-01-21T22:52:54.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.049+0000 7f1b51000640 1 -- 172.21.15.138:0/2275961306 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4c0c16f0 con 0x7f1b4c0ca9e0 2024-01-21T22:52:54.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.051+0000 7f1b51000640 1 -- 172.21.15.138:0/2275961306 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b4c0c1ef0 con 0x7f1b4c0ca9e0 2024-01-21T22:52:54.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.303+0000 7f1b43400640 1 -- 172.21.15.138:0/2275961306 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f1b4403b050 con 0x7f1b4c0ca9e0 2024-01-21T22:52:54.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.306+0000 7f1b43400640 1 --2- 172.21.15.138:0/2275961306 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f1b2406b380 0x7f1b2406d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:54.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.306+0000 7f1b4b400640 1 --2- 172.21.15.138:0/2275961306 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f1b2406b380 0x7f1b2406d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:54.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.307+0000 7f1b43400640 1 -- 172.21.15.138:0/2275961306 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1b4407aff0 con 0x7f1b4c0ca9e0 2024-01-21T22:52:54.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.307+0000 7f1b4b400640 1 --2- 172.21.15.138:0/2275961306 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f1b2406b380 0x7f1b2406d840 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c009a20 tx=0x7f1b3c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:54.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.318+0000 7f1b43400640 1 -- 172.21.15.138:0/2275961306 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f1b44077330 con 0x7f1b4c0ca9e0 2024-01-21T22:52:54.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.525+0000 7f1b51000640 1 -- 172.21.15.138:0/2275961306 --> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b4c0cd910 con 0x7f1b2406b380 2024-01-21T22:52:54.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.526+0000 7f1b43400640 1 -- 172.21.15.138:0/2275961306 <== mgr.4111 v2:172.21.15.152:6800/314625564 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1b3c004020 con 0x7f1b2406b380 2024-01-21T22:52:54.527 INFO:teuthology.orchestra.run.smithi138.stderr:instructed osd(s) 7 to deep-scrub 2024-01-21T22:52:54.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.529+0000 7f1b51000640 1 -- 172.21.15.138:0/2275961306 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f1b2406b380 msgr2=0x7f1b2406d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:54.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.529+0000 7f1b51000640 1 --2- 172.21.15.138:0/2275961306 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f1b2406b380 0x7f1b2406d840 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c009a20 tx=0x7f1b3c009190 comp rx=0 tx=0).stop 2024-01-21T22:52:54.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.529+0000 7f1b51000640 1 -- 172.21.15.138:0/2275961306 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b4c0ca9e0 msgr2=0x7f1b4c148770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:54.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.529+0000 7f1b51000640 1 --2- 172.21.15.138:0/2275961306 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b4c0ca9e0 0x7f1b4c148770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c155390 tx=0x7f1b44004a40 comp rx=0 tx=0).stop 2024-01-21T22:52:54.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.529+0000 7f1b51000640 1 -- 172.21.15.138:0/2275961306 shutdown_connections 2024-01-21T22:52:54.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.529+0000 7f1b51000640 1 --2- 172.21.15.138:0/2275961306 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f1b2406b380 0x7f1b2406d840 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:54.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.529+0000 7f1b51000640 1 --2- 172.21.15.138:0/2275961306 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f1b4c0ca9e0 0x7f1b4c148770 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:54.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.529+0000 7f1b51000640 1 --2- 172.21.15.138:0/2275961306 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f1b4c0c7be0 0x7f1b4c0c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:54.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.529+0000 7f1b51000640 1 --2- 172.21.15.138:0/2275961306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1b4c066b60 0x7f1b4c0c21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:54.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.529+0000 7f1b51000640 1 -- 172.21.15.138:0/2275961306 >> 172.21.15.138:0/2275961306 conn(0x7f1b4c06a8d0 msgr2=0x7f1b4c06e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:54.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.529+0000 7f1b51000640 1 -- 172.21.15.138:0/2275961306 shutdown_connections 2024-01-21T22:52:54.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.529+0000 7f1b51000640 1 -- 172.21.15.138:0/2275961306 wait complete. 2024-01-21T22:52:54.547 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-21T22:52:54.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.747+0000 7f953ac00640 1 Processor -- start 2024-01-21T22:52:54.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.747+0000 7f953ac00640 1 -- start start 2024-01-21T22:52:54.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.748+0000 7f953ac00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f95340716e0 0x7f95340c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:54.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.748+0000 7f953ac00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f95340c7520 0x7f953406ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:54.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.748+0000 7f953ac00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9534157740 con 0x7f9534150a00 2024-01-21T22:52:54.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.748+0000 7f953ac00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95340c4cc0 con 0x7f95340c7520 2024-01-21T22:52:54.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.748+0000 7f953ac00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f953406a920 con 0x7f95340716e0 2024-01-21T22:52:54.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.750+0000 7f9533e00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f9534150a00 0x7f95340c25b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:34298/0 (socket says 172.21.15.138:34298) 2024-01-21T22:52:54.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.750+0000 7f9533e00640 1 -- 172.21.15.138:0/2478574779 learned_addr learned my addr 172.21.15.138:0/2478574779 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:52:54.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.754+0000 7f952b200640 1 --2- 172.21.15.138:0/2478574779 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f95340716e0 0x7f95340c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:54.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.760+0000 7f9538c00640 1 --2- 172.21.15.138:0/2478574779 >> v2:172.21.15.138:3300/0 conn(0x7f95340c7520 0x7f953406ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:54.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.760+0000 7f952b200640 1 -- 172.21.15.138:0/2478574779 >> v2:172.21.15.138:3300/0 conn(0x7f95340c7520 msgr2=0x7f953406ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:54.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.760+0000 7f952b200640 1 --2- 172.21.15.138:0/2478574779 >> v2:172.21.15.138:3300/0 conn(0x7f95340c7520 0x7f953406ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:54.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.760+0000 7f952b200640 1 -- 172.21.15.138:0/2478574779 >> v1:172.21.15.152:6789/0 conn(0x7f9534150a00 legacy=0x7f95340c25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:52:54.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.760+0000 7f952b200640 1 -- 172.21.15.138:0/2478574779 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9534156bc0 con 0x7f95340716e0 2024-01-21T22:52:54.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.763+0000 7f952b200640 1 --2- 172.21.15.138:0/2478574779 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f95340716e0 0x7f95340c6fe0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f953406d3c0 tx=0x7f951c02d970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=761ae8ce68eab895 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:54.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.846+0000 7f9533400640 1 -- 172.21.15.138:0/2478574779 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c032830 con 0x7f95340716e0 2024-01-21T22:52:54.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.846+0000 7f9533400640 1 -- 172.21.15.138:0/2478574779 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951c032990 con 0x7f95340716e0 2024-01-21T22:52:54.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.846+0000 7f9533400640 1 -- 172.21.15.138:0/2478574779 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c032d20 con 0x7f95340716e0 2024-01-21T22:52:54.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.847+0000 7f953ac00640 1 -- 172.21.15.138:0/2478574779 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f95340716e0 msgr2=0x7f95340c6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:54.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.847+0000 7f953ac00640 1 --2- 172.21.15.138:0/2478574779 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f95340716e0 0x7f95340c6fe0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f953406d3c0 tx=0x7f951c02d970 comp rx=0 tx=0).stop 2024-01-21T22:52:54.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.847+0000 7f953ac00640 1 -- 172.21.15.138:0/2478574779 shutdown_connections 2024-01-21T22:52:54.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.847+0000 7f953ac00640 1 --2- 172.21.15.138:0/2478574779 >> v2:172.21.15.138:3300/0 conn(0x7f95340c7520 0x7f953406ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:54.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.847+0000 7f953ac00640 1 --2- 172.21.15.138:0/2478574779 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f95340716e0 0x7f95340c6fe0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:54.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.847+0000 7f953ac00640 1 -- 172.21.15.138:0/2478574779 >> 172.21.15.138:0/2478574779 conn(0x7f95340cb0b0 msgr2=0x7f95340cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:54.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.847+0000 7f953ac00640 1 -- 172.21.15.138:0/2478574779 shutdown_connections 2024-01-21T22:52:54.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.848+0000 7f953ac00640 1 -- 172.21.15.138:0/2478574779 wait complete. 2024-01-21T22:52:54.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.850+0000 7f953ac00640 1 Processor -- start 2024-01-21T22:52:54.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.851+0000 7f953ac00640 1 -- start start 2024-01-21T22:52:54.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.852+0000 7f953ac00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f95340716e0 0x7f95340ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:54.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.853+0000 7f953ac00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95340c7520 0x7f95341ef5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:54.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.853+0000 7f953ac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9534150a00 0x7f9534143870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:54.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.853+0000 7f953ac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95341512d0 con 0x7f9534150a00 2024-01-21T22:52:54.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.853+0000 7f953ac00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95340c12c0 con 0x7f95340c7520 2024-01-21T22:52:54.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.853+0000 7f953ac00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95341f3a50 con 0x7f95340716e0 2024-01-21T22:52:54.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.854+0000 7f952b200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95340c7520 0x7f95341ef5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:54.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.854+0000 7f9533e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f95340716e0 0x7f95340ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:54.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.854+0000 7f952b200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95340c7520 0x7f95341ef5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:39330/0 (socket says 172.21.15.138:39330) 2024-01-21T22:52:54.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.854+0000 7f952b200640 1 -- 172.21.15.138:0/389856380 learned_addr learned my addr 172.21.15.138:0/389856380 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:52:54.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.857+0000 7f9533e00640 1 -- 172.21.15.138:0/389856380 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95340c7520 msgr2=0x7f95341ef5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:54.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.857+0000 7f9533e00640 1 --2- 172.21.15.138:0/389856380 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95340c7520 0x7f95341ef5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:54.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.857+0000 7f9533e00640 1 -- 172.21.15.138:0/389856380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9534150a00 msgr2=0x7f9534143870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:54.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.857+0000 7f9533e00640 1 --2- 172.21.15.138:0/389856380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9534150a00 0x7f9534143870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:54.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.857+0000 7f9533e00640 1 -- 172.21.15.138:0/389856380 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c009400 con 0x7f95340716e0 2024-01-21T22:52:54.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.860+0000 7f9533e00640 1 --2- 172.21.15.138:0/389856380 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f95340716e0 0x7f95340ca3a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f952400bd30 tx=0x7f952400be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:54.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.862+0000 7f9530c00640 1 -- 172.21.15.138:0/389856380 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524011710 con 0x7f95340716e0 2024-01-21T22:52:54.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.862+0000 7f953ac00640 1 -- 172.21.15.138:0/389856380 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95340c0d20 con 0x7f95340716e0 2024-01-21T22:52:54.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.862+0000 7f953ac00640 1 -- 172.21.15.138:0/389856380 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95341f2c90 con 0x7f95340716e0 2024-01-21T22:52:54.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.863+0000 7f953ac00640 1 -- 172.21.15.138:0/389856380 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f953404eaa0 con 0x7f95340716e0 2024-01-21T22:52:54.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.864+0000 7f9530c00640 1 -- 172.21.15.138:0/389856380 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952400f050 con 0x7f95340716e0 2024-01-21T22:52:54.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.864+0000 7f9530c00640 1 -- 172.21.15.138:0/389856380 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524011a70 con 0x7f95340716e0 2024-01-21T22:52:54.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.884+0000 7f9530c00640 1 -- 172.21.15.138:0/389856380 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f9524011bd0 con 0x7f95340716e0 2024-01-21T22:52:54.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.885+0000 7f9530c00640 1 --2- 172.21.15.138:0/389856380 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f950c06b360 0x7f950c06d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:52:54.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.886+0000 7f952b200640 1 --2- 172.21.15.138:0/389856380 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f950c06b360 0x7f950c06d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:52:54.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.886+0000 7f9530c00640 1 -- 172.21.15.138:0/389856380 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f952405a1e0 con 0x7f95340716e0 2024-01-21T22:52:54.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:54.887+0000 7f952b200640 1 --2- 172.21.15.138:0/389856380 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f950c06b360 0x7f950c06d820 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f95340c2e60 tx=0x7f951c03d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:52:55.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:55.126+0000 7f9530c00640 1 -- 172.21.15.138:0/389856380 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f95240598b0 con 0x7f95340716e0 2024-01-21T22:52:55.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:55.331+0000 7f953ac00640 1 -- 172.21.15.138:0/389856380 --> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9534157460 con 0x7f950c06b360 2024-01-21T22:52:55.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:55.340+0000 7f9530c00640 1 -- 172.21.15.138:0/389856380 <== mgr.4111 v2:172.21.15.152:6800/314625564 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287640 (secure 0 0 0) 0x7f951c032c00 con 0x7f950c06b360 2024-01-21T22:52:55.345 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:52:55.346 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-01-21T22:52:55.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:55.347+0000 7f953ac00640 1 -- 172.21.15.138:0/389856380 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f950c06b360 msgr2=0x7f950c06d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:55.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:55.347+0000 7f953ac00640 1 --2- 172.21.15.138:0/389856380 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f950c06b360 0x7f950c06d820 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f95340c2e60 tx=0x7f951c03d040 comp rx=0 tx=0).stop 2024-01-21T22:52:55.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:55.347+0000 7f953ac00640 1 -- 172.21.15.138:0/389856380 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f95340716e0 msgr2=0x7f95340ca3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:52:55.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:55.347+0000 7f953ac00640 1 --2- 172.21.15.138:0/389856380 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f95340716e0 0x7f95340ca3a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f952400bd30 tx=0x7f952400be20 comp rx=0 tx=0).stop 2024-01-21T22:52:55.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:55.347+0000 7f953ac00640 1 -- 172.21.15.138:0/389856380 shutdown_connections 2024-01-21T22:52:55.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:55.347+0000 7f953ac00640 1 --2- 172.21.15.138:0/389856380 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f950c06b360 0x7f950c06d820 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:55.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:55.347+0000 7f953ac00640 1 --2- 172.21.15.138:0/389856380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9534150a00 0x7f9534143870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:55.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:55.347+0000 7f953ac00640 1 --2- 172.21.15.138:0/389856380 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f95340c7520 0x7f95341ef5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:55.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:55.347+0000 7f953ac00640 1 --2- 172.21.15.138:0/389856380 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f95340716e0 0x7f95340ca3a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:52:55.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:55.347+0000 7f953ac00640 1 -- 172.21.15.138:0/389856380 >> 172.21.15.138:0/389856380 conn(0x7f95340cb0b0 msgr2=0x7f953406c890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:52:55.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:55.347+0000 7f953ac00640 1 -- 172.21.15.138:0/389856380 shutdown_connections 2024-01-21T22:52:55.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:52:55.347+0000 7f953ac00640 1 -- 172.21.15.138:0/389856380 wait complete. 2024-01-21T22:52:55.368 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":864,"stamp":"2024-01-21T22:52:53.539956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":82810866,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":1695,"num_read_kb":212798,"num_write":26823,"num_write_kb":599405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28172,"ondisk_log_size":28172,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":610384,"kb_used_data":163856,"kb_used_omap":12,"kb_used_meta":446451,"kb_avail":754364336,"statfs":{"total":773094113280,"available":772469080064,"internally_reserved":0,"allocated":167788544,"data_stored":166397260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":457166385},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23135,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":3,"num_write":16,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002182"},"pg_stats":[{"pgid":"3.3f","version":"25'165","reported_seq":201,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.429045+0000","last_change":"2024-01-21T22:25:04.888366+0000","last_active":"2024-01-21T22:52:41.429045+0000","last_peered":"2024-01-21T22:52:41.429045+0000","last_clean":"2024-01-21T22:52:41.429045+0000","last_became_active":"2024-01-21T22:25:04.888060+0000","last_became_peered":"2024-01-21T22:25:04.888060+0000","last_unstale":"2024-01-21T22:52:41.429045+0000","last_undegraded":"2024-01-21T22:52:41.429045+0000","last_fullsized":"2024-01-21T22:52:41.429045+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":115,"num_write_kb":6171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"25'170","reported_seq":211,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.418894+0000","last_change":"2024-01-21T22:25:04.843704+0000","last_active":"2024-01-21T22:52:41.418894+0000","last_peered":"2024-01-21T22:52:41.418894+0000","last_clean":"2024-01-21T22:52:41.418894+0000","last_became_active":"2024-01-21T22:25:04.843471+0000","last_became_peered":"2024-01-21T22:25:04.843471+0000","last_unstale":"2024-01-21T22:52:41.418894+0000","last_undegraded":"2024-01-21T22:52:41.418894+0000","last_fullsized":"2024-01-21T22:52:41.418894+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":129,"num_write_kb":4796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"25'202","reported_seq":242,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.418566+0000","last_change":"2024-01-21T22:25:04.858021+0000","last_active":"2024-01-21T22:52:41.418566+0000","last_peered":"2024-01-21T22:52:41.418566+0000","last_clean":"2024-01-21T22:52:41.418566+0000","last_became_active":"2024-01-21T22:25:04.857685+0000","last_became_peered":"2024-01-21T22:25:04.857685+0000","last_unstale":"2024-01-21T22:52:41.418566+0000","last_undegraded":"2024-01-21T22:52:41.418566+0000","last_fullsized":"2024-01-21T22:52:41.418566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":157,"num_write_kb":6223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"25'180","reported_seq":224,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:53.105288+0000","last_change":"2024-01-21T22:52:53.105348+0000","last_active":"2024-01-21T22:52:53.105288+0000","last_peered":"2024-01-21T22:52:53.105288+0000","last_clean":"2024-01-21T22:52:53.105288+0000","last_became_active":"2024-01-21T22:25:04.858278+0000","last_became_peered":"2024-01-21T22:25:04.858278+0000","last_unstale":"2024-01-21T22:52:53.105288+0000","last_undegraded":"2024-01-21T22:52:53.105288+0000","last_fullsized":"2024-01-21T22:52:53.105288+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'180","last_scrub_stamp":"2024-01-21T22:52:53.105100+0000","last_deep_scrub":"25'180","last_deep_scrub_stamp":"2024-01-21T22:52:53.105100+0000","last_clean_scrub_stamp":"2024-01-21T22:52:53.105100+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:43:27.094985+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":135,"num_write_kb":19244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"25'181","reported_seq":219,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.430068+0000","last_change":"2024-01-21T22:25:04.881014+0000","last_active":"2024-01-21T22:52:41.430068+0000","last_peered":"2024-01-21T22:52:41.430068+0000","last_clean":"2024-01-21T22:52:41.430068+0000","last_became_active":"2024-01-21T22:25:04.878457+0000","last_became_peered":"2024-01-21T22:25:04.878457+0000","last_unstale":"2024-01-21T22:52:41.430068+0000","last_undegraded":"2024-01-21T22:52:41.430068+0000","last_fullsized":"2024-01-21T22:52:41.430068+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":119,"num_write_kb":7041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"25'199","reported_seq":228,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.361814+0000","last_change":"2024-01-21T22:25:04.859776+0000","last_active":"2024-01-21T22:52:41.361814+0000","last_peered":"2024-01-21T22:52:41.361814+0000","last_clean":"2024-01-21T22:52:41.361814+0000","last_became_active":"2024-01-21T22:25:04.858731+0000","last_became_peered":"2024-01-21T22:25:04.858731+0000","last_unstale":"2024-01-21T22:52:41.361814+0000","last_undegraded":"2024-01-21T22:52:41.361814+0000","last_fullsized":"2024-01-21T22:52:41.361814+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":148,"num_write_kb":2956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"25'212","reported_seq":257,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.427912+0000","last_change":"2024-01-21T22:25:04.881392+0000","last_active":"2024-01-21T22:52:41.427912+0000","last_peered":"2024-01-21T22:52:41.427912+0000","last_clean":"2024-01-21T22:52:41.427912+0000","last_became_active":"2024-01-21T22:25:04.881090+0000","last_became_peered":"2024-01-21T22:25:04.881090+0000","last_unstale":"2024-01-21T22:52:41.427912+0000","last_undegraded":"2024-01-21T22:52:41.427912+0000","last_fullsized":"2024-01-21T22:52:41.427912+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":4572,"num_write":149,"num_write_kb":5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"25'174","reported_seq":214,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.422882+0000","last_change":"2024-01-21T22:25:04.879186+0000","last_active":"2024-01-21T22:52:41.422882+0000","last_peered":"2024-01-21T22:52:41.422882+0000","last_clean":"2024-01-21T22:52:41.422882+0000","last_became_active":"2024-01-21T22:25:04.878824+0000","last_became_peered":"2024-01-21T22:25:04.878824+0000","last_unstale":"2024-01-21T22:52:41.422882+0000","last_undegraded":"2024-01-21T22:52:41.422882+0000","last_fullsized":"2024-01-21T22:52:41.422882+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":137,"num_write_kb":11909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"25'162","reported_seq":196,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.446005+0000","last_change":"2024-01-21T22:25:04.886391+0000","last_active":"2024-01-21T22:52:41.446005+0000","last_peered":"2024-01-21T22:52:41.446005+0000","last_clean":"2024-01-21T22:52:41.446005+0000","last_became_active":"2024-01-21T22:25:04.884979+0000","last_became_peered":"2024-01-21T22:25:04.884979+0000","last_unstale":"2024-01-21T22:52:41.446005+0000","last_undegraded":"2024-01-21T22:52:41.446005+0000","last_fullsized":"2024-01-21T22:52:41.446005+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":96,"num_write_kb":3924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"25'180","reported_seq":220,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.430495+0000","last_change":"2024-01-21T22:25:04.888126+0000","last_active":"2024-01-21T22:52:41.430495+0000","last_peered":"2024-01-21T22:52:41.430495+0000","last_clean":"2024-01-21T22:52:41.430495+0000","last_became_active":"2024-01-21T22:25:04.887631+0000","last_became_peered":"2024-01-21T22:25:04.887631+0000","last_unstale":"2024-01-21T22:52:41.430495+0000","last_undegraded":"2024-01-21T22:52:41.430495+0000","last_fullsized":"2024-01-21T22:52:41.430495+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":122,"num_write_kb":9871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"25'192","reported_seq":227,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.429439+0000","last_change":"2024-01-21T22:25:04.885677+0000","last_active":"2024-01-21T22:52:41.429439+0000","last_peered":"2024-01-21T22:52:41.429439+0000","last_clean":"2024-01-21T22:52:41.429439+0000","last_became_active":"2024-01-21T22:25:04.884697+0000","last_became_peered":"2024-01-21T22:25:04.884697+0000","last_unstale":"2024-01-21T22:52:41.429439+0000","last_undegraded":"2024-01-21T22:52:41.429439+0000","last_fullsized":"2024-01-21T22:52:41.429439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":143,"num_write_kb":7172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"25'146","reported_seq":184,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.434001+0000","last_change":"2024-01-21T22:25:04.886799+0000","last_active":"2024-01-21T22:52:41.434001+0000","last_peered":"2024-01-21T22:52:41.434001+0000","last_clean":"2024-01-21T22:52:41.434001+0000","last_became_active":"2024-01-21T22:25:04.886387+0000","last_became_peered":"2024-01-21T22:25:04.886387+0000","last_unstale":"2024-01-21T22:52:41.434001+0000","last_undegraded":"2024-01-21T22:52:41.434001+0000","last_fullsized":"2024-01-21T22:52:41.434001+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":92,"num_write_kb":3768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"25'169","reported_seq":202,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.350337+0000","last_change":"2024-01-21T22:25:04.888103+0000","last_active":"2024-01-21T22:52:41.350337+0000","last_peered":"2024-01-21T22:52:41.350337+0000","last_clean":"2024-01-21T22:52:41.350337+0000","last_became_active":"2024-01-21T22:25:04.887908+0000","last_became_peered":"2024-01-21T22:25:04.887908+0000","last_unstale":"2024-01-21T22:52:41.350337+0000","last_undegraded":"2024-01-21T22:52:41.350337+0000","last_fullsized":"2024-01-21T22:52:41.350337+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:23:52.444277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":122,"num_write_kb":8957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"25'174","reported_seq":213,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.343215+0000","last_change":"2024-01-21T22:25:04.842750+0000","last_active":"2024-01-21T22:52:41.343215+0000","last_peered":"2024-01-21T22:52:41.343215+0000","last_clean":"2024-01-21T22:52:41.343215+0000","last_became_active":"2024-01-21T22:25:04.842512+0000","last_became_peered":"2024-01-21T22:25:04.842512+0000","last_unstale":"2024-01-21T22:52:41.343215+0000","last_undegraded":"2024-01-21T22:52:41.343215+0000","last_fullsized":"2024-01-21T22:52:41.343215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:40:05.554704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":120,"num_write_kb":6061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"25'170","reported_seq":227,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.420973+0000","last_change":"2024-01-21T22:25:04.876592+0000","last_active":"2024-01-21T22:52:41.420973+0000","last_peered":"2024-01-21T22:52:41.420973+0000","last_clean":"2024-01-21T22:52:41.420973+0000","last_became_active":"2024-01-21T22:25:04.876305+0000","last_became_peered":"2024-01-21T22:25:04.876305+0000","last_unstale":"2024-01-21T22:52:41.420973+0000","last_undegraded":"2024-01-21T22:52:41.420973+0000","last_fullsized":"2024-01-21T22:52:41.420973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":7620,"num_write":119,"num_write_kb":10742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"25'209","reported_seq":256,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.415609+0000","last_change":"2024-01-21T22:25:04.878038+0000","last_active":"2024-01-21T22:52:41.415609+0000","last_peered":"2024-01-21T22:52:41.415609+0000","last_clean":"2024-01-21T22:52:41.415609+0000","last_became_active":"2024-01-21T22:25:04.877716+0000","last_became_peered":"2024-01-21T22:25:04.877716+0000","last_unstale":"2024-01-21T22:52:41.415609+0000","last_undegraded":"2024-01-21T22:52:41.415609+0000","last_fullsized":"2024-01-21T22:52:41.415609+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":5080,"num_write":159,"num_write_kb":7672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"25'183","reported_seq":221,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.403246+0000","last_change":"2024-01-21T22:25:04.890537+0000","last_active":"2024-01-21T22:52:41.403246+0000","last_peered":"2024-01-21T22:52:41.403246+0000","last_clean":"2024-01-21T22:52:41.403246+0000","last_became_active":"2024-01-21T22:25:04.890298+0000","last_became_peered":"2024-01-21T22:25:04.890298+0000","last_unstale":"2024-01-21T22:52:41.403246+0000","last_undegraded":"2024-01-21T22:52:41.403246+0000","last_fullsized":"2024-01-21T22:52:41.403246+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":136,"num_write_kb":7248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"25'159","reported_seq":194,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.349490+0000","last_change":"2024-01-21T22:25:04.712868+0000","last_active":"2024-01-21T22:52:41.349490+0000","last_peered":"2024-01-21T22:52:41.349490+0000","last_clean":"2024-01-21T22:52:41.349490+0000","last_became_active":"2024-01-21T22:25:04.712616+0000","last_became_peered":"2024-01-21T22:25:04.712616+0000","last_unstale":"2024-01-21T22:52:41.349490+0000","last_undegraded":"2024-01-21T22:52:41.349490+0000","last_fullsized":"2024-01-21T22:52:41.349490+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:50:41.306582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":102,"num_write_kb":4431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"25'163","reported_seq":204,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.410525+0000","last_change":"2024-01-21T22:25:04.879328+0000","last_active":"2024-01-21T22:52:41.410525+0000","last_peered":"2024-01-21T22:52:41.410525+0000","last_clean":"2024-01-21T22:52:41.410525+0000","last_became_active":"2024-01-21T22:25:04.879011+0000","last_became_peered":"2024-01-21T22:25:04.879011+0000","last_unstale":"2024-01-21T22:52:41.410525+0000","last_undegraded":"2024-01-21T22:52:41.410525+0000","last_fullsized":"2024-01-21T22:52:41.410525+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":108,"num_write_kb":6073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"25'243","reported_seq":285,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.434256+0000","last_change":"2024-01-21T22:25:04.885620+0000","last_active":"2024-01-21T22:52:41.434256+0000","last_peered":"2024-01-21T22:52:41.434256+0000","last_clean":"2024-01-21T22:52:41.434256+0000","last_became_active":"2024-01-21T22:25:04.884364+0000","last_became_peered":"2024-01-21T22:25:04.884364+0000","last_unstale":"2024-01-21T22:52:41.434256+0000","last_undegraded":"2024-01-21T22:52:41.434256+0000","last_fullsized":"2024-01-21T22:52:41.434256+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3556,"num_write":191,"num_write_kb":16584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"25'200","reported_seq":240,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.445510+0000","last_change":"2024-01-21T22:25:04.868371+0000","last_active":"2024-01-21T22:52:41.445510+0000","last_peered":"2024-01-21T22:52:41.445510+0000","last_clean":"2024-01-21T22:52:41.445510+0000","last_became_active":"2024-01-21T22:25:04.868196+0000","last_became_peered":"2024-01-21T22:25:04.868196+0000","last_unstale":"2024-01-21T22:52:41.445510+0000","last_undegraded":"2024-01-21T22:52:41.445510+0000","last_fullsized":"2024-01-21T22:52:41.445510+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":139,"num_write_kb":3732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"25'203","reported_seq":269,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:52.935450+0000","last_change":"2024-01-21T22:52:52.935491+0000","last_active":"2024-01-21T22:52:52.935450+0000","last_peered":"2024-01-21T22:52:52.935450+0000","last_clean":"2024-01-21T22:52:52.935450+0000","last_became_active":"2024-01-21T22:25:04.878868+0000","last_became_peered":"2024-01-21T22:25:04.878868+0000","last_unstale":"2024-01-21T22:52:52.935450+0000","last_undegraded":"2024-01-21T22:52:52.935450+0000","last_fullsized":"2024-01-21T22:52:52.935450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'203","last_scrub_stamp":"2024-01-21T22:52:52.935365+0000","last_deep_scrub":"25'203","last_deep_scrub_stamp":"2024-01-21T22:52:52.935365+0000","last_clean_scrub_stamp":"2024-01-21T22:52:52.935365+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:46:06.748617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":7620,"num_write":153,"num_write_kb":6732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"25'173","reported_seq":210,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.419999+0000","last_change":"2024-01-21T22:25:04.715246+0000","last_active":"2024-01-21T22:52:41.419999+0000","last_peered":"2024-01-21T22:52:41.419999+0000","last_clean":"2024-01-21T22:52:41.419999+0000","last_became_active":"2024-01-21T22:25:04.714969+0000","last_became_peered":"2024-01-21T22:25:04.714969+0000","last_unstale":"2024-01-21T22:52:41.419999+0000","last_undegraded":"2024-01-21T22:52:41.419999+0000","last_fullsized":"2024-01-21T22:52:41.419999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":106,"num_write_kb":4123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"25'182","reported_seq":231,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.403668+0000","last_change":"2024-01-21T22:25:04.877902+0000","last_active":"2024-01-21T22:52:41.403668+0000","last_peered":"2024-01-21T22:52:41.403668+0000","last_clean":"2024-01-21T22:52:41.403668+0000","last_became_active":"2024-01-21T22:25:04.876963+0000","last_became_peered":"2024-01-21T22:25:04.876963+0000","last_unstale":"2024-01-21T22:52:41.403668+0000","last_undegraded":"2024-01-21T22:52:41.403668+0000","last_fullsized":"2024-01-21T22:52:41.403668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":5334,"num_write":137,"num_write_kb":8161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"25'184","reported_seq":235,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.419553+0000","last_change":"2024-01-21T22:25:04.716735+0000","last_active":"2024-01-21T22:52:41.419553+0000","last_peered":"2024-01-21T22:52:41.419553+0000","last_clean":"2024-01-21T22:52:41.419553+0000","last_became_active":"2024-01-21T22:25:04.716570+0000","last_became_peered":"2024-01-21T22:25:04.716570+0000","last_unstale":"2024-01-21T22:52:41.419553+0000","last_undegraded":"2024-01-21T22:52:41.419553+0000","last_fullsized":"2024-01-21T22:52:41.419553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":5842,"num_write":145,"num_write_kb":5757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"25'176","reported_seq":218,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.368987+0000","last_change":"2024-01-21T22:25:04.886230+0000","last_active":"2024-01-21T22:52:41.368987+0000","last_peered":"2024-01-21T22:52:41.368987+0000","last_clean":"2024-01-21T22:52:41.368987+0000","last_became_active":"2024-01-21T22:25:04.885523+0000","last_became_peered":"2024-01-21T22:25:04.885523+0000","last_unstale":"2024-01-21T22:52:41.368987+0000","last_undegraded":"2024-01-21T22:52:41.368987+0000","last_fullsized":"2024-01-21T22:52:41.368987+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3556,"num_write":106,"num_write_kb":5820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"25'213","reported_seq":249,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.346332+0000","last_change":"2024-01-21T22:25:04.842692+0000","last_active":"2024-01-21T22:52:41.346332+0000","last_peered":"2024-01-21T22:52:41.346332+0000","last_clean":"2024-01-21T22:52:41.346332+0000","last_became_active":"2024-01-21T22:25:04.842429+0000","last_became_peered":"2024-01-21T22:25:04.842429+0000","last_unstale":"2024-01-21T22:52:41.346332+0000","last_undegraded":"2024-01-21T22:52:41.346332+0000","last_fullsized":"2024-01-21T22:52:41.346332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:52:24.233709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":167,"num_write_kb":10319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"25'183","reported_seq":222,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.350241+0000","last_change":"2024-01-21T22:25:04.888296+0000","last_active":"2024-01-21T22:52:41.350241+0000","last_peered":"2024-01-21T22:52:41.350241+0000","last_clean":"2024-01-21T22:52:41.350241+0000","last_became_active":"2024-01-21T22:25:04.888025+0000","last_became_peered":"2024-01-21T22:25:04.888025+0000","last_unstale":"2024-01-21T22:52:41.350241+0000","last_undegraded":"2024-01-21T22:52:41.350241+0000","last_fullsized":"2024-01-21T22:52:41.350241+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:57:44.129490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":136,"num_write_kb":6753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"25'199","reported_seq":240,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.417851+0000","last_change":"2024-01-21T22:25:04.711908+0000","last_active":"2024-01-21T22:52:41.417851+0000","last_peered":"2024-01-21T22:52:41.417851+0000","last_clean":"2024-01-21T22:52:41.417851+0000","last_became_active":"2024-01-21T22:25:04.711654+0000","last_became_peered":"2024-01-21T22:25:04.711654+0000","last_unstale":"2024-01-21T22:52:41.417851+0000","last_undegraded":"2024-01-21T22:52:41.417851+0000","last_fullsized":"2024-01-21T22:52:41.417851+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":148,"num_write_kb":7328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"25'334","reported_seq":376,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.406539+0000","last_change":"2024-01-21T22:25:04.876757+0000","last_active":"2024-01-21T22:52:41.406539+0000","last_peered":"2024-01-21T22:52:41.406539+0000","last_clean":"2024-01-21T22:52:41.406539+0000","last_became_active":"2024-01-21T22:25:04.876012+0000","last_became_peered":"2024-01-21T22:25:04.876012+0000","last_unstale":"2024-01-21T22:52:41.406539+0000","last_undegraded":"2024-01-21T22:52:41.406539+0000","last_fullsized":"2024-01-21T22:52:41.406539+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":275,"num_write_kb":7140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"25'207","reported_seq":243,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.342299+0000","last_change":"2024-01-21T22:25:04.715277+0000","last_active":"2024-01-21T22:52:41.342299+0000","last_peered":"2024-01-21T22:52:41.342299+0000","last_clean":"2024-01-21T22:52:41.342299+0000","last_became_active":"2024-01-21T22:25:04.715034+0000","last_became_peered":"2024-01-21T22:25:04.715034+0000","last_unstale":"2024-01-21T22:52:41.342299+0000","last_undegraded":"2024-01-21T22:52:41.342299+0000","last_fullsized":"2024-01-21T22:52:41.342299+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:55:28.069411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":140,"num_write_kb":6049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"25'210","reported_seq":251,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:51.121042+0000","last_change":"2024-01-21T22:52:51.121096+0000","last_active":"2024-01-21T22:52:51.121042+0000","last_peered":"2024-01-21T22:52:51.121042+0000","last_clean":"2024-01-21T22:52:51.121042+0000","last_became_active":"2024-01-21T22:25:04.843827+0000","last_became_peered":"2024-01-21T22:25:04.843827+0000","last_unstale":"2024-01-21T22:52:51.121042+0000","last_undegraded":"2024-01-21T22:52:51.121042+0000","last_fullsized":"2024-01-21T22:52:51.121042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'210","last_scrub_stamp":"2024-01-21T22:52:51.120891+0000","last_deep_scrub":"25'210","last_deep_scrub_stamp":"2024-01-21T22:52:51.120891+0000","last_clean_scrub_stamp":"2024-01-21T22:52:51.120891+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:06:29.439012+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1016,"num_write":149,"num_write_kb":12027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"25'158","reported_seq":204,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:50.887891+0000","last_change":"2024-01-21T22:52:50.887999+0000","last_active":"2024-01-21T22:52:50.887891+0000","last_peered":"2024-01-21T22:52:50.887891+0000","last_clean":"2024-01-21T22:52:50.887891+0000","last_became_active":"2024-01-21T22:25:04.712814+0000","last_became_peered":"2024-01-21T22:25:04.712814+0000","last_unstale":"2024-01-21T22:52:50.887891+0000","last_undegraded":"2024-01-21T22:52:50.887891+0000","last_fullsized":"2024-01-21T22:52:50.887891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'158","last_scrub_stamp":"2024-01-21T22:52:50.887622+0000","last_deep_scrub":"25'158","last_deep_scrub_stamp":"2024-01-21T22:52:50.887622+0000","last_clean_scrub_stamp":"2024-01-21T22:52:50.887622+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:13:27.478094+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":111,"num_write_kb":4009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"25'754","reported_seq":790,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:52.391023+0000","last_change":"2024-01-21T22:52:52.391050+0000","last_active":"2024-01-21T22:52:52.391023+0000","last_peered":"2024-01-21T22:52:52.391023+0000","last_clean":"2024-01-21T22:52:52.391023+0000","last_became_active":"2024-01-21T22:25:03.702776+0000","last_became_peered":"2024-01-21T22:25:03.702776+0000","last_unstale":"2024-01-21T22:52:52.391023+0000","last_undegraded":"2024-01-21T22:52:52.391023+0000","last_fullsized":"2024-01-21T22:52:52.391023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'754","last_scrub_stamp":"2024-01-21T22:52:52.390950+0000","last_deep_scrub":"25'754","last_deep_scrub_stamp":"2024-01-21T22:52:52.390950+0000","last_clean_scrub_stamp":"2024-01-21T22:52:52.390950+0000","objects_scrubbed":1,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:17:20.739873+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":748,"num_write_kb":4569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"25'198","reported_seq":236,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.445864+0000","last_change":"2024-01-21T22:25:04.877158+0000","last_active":"2024-01-21T22:52:41.445864+0000","last_peered":"2024-01-21T22:52:41.445864+0000","last_clean":"2024-01-21T22:52:41.445864+0000","last_became_active":"2024-01-21T22:25:04.876920+0000","last_became_peered":"2024-01-21T22:25:04.876920+0000","last_unstale":"2024-01-21T22:52:41.445864+0000","last_undegraded":"2024-01-21T22:52:41.445864+0000","last_fullsized":"2024-01-21T22:52:41.445864+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":152,"num_write_kb":6158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"25'776","reported_seq":804,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.530416+0000","last_change":"2024-01-21T22:25:03.720178+0000","last_active":"2024-01-21T22:52:33.530416+0000","last_peered":"2024-01-21T22:52:33.530416+0000","last_clean":"2024-01-21T22:52:33.530416+0000","last_became_active":"2024-01-21T22:25:03.719909+0000","last_became_peered":"2024-01-21T22:25:03.719909+0000","last_unstale":"2024-01-21T22:52:33.530416+0000","last_undegraded":"2024-01-21T22:52:33.530416+0000","last_fullsized":"2024-01-21T22:52:33.530416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":769,"num_write_kb":4580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"25'3","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.462701+0000","last_change":"2024-01-21T22:25:03.723005+0000","last_active":"2024-01-21T22:52:28.462701+0000","last_peered":"2024-01-21T22:52:28.462701+0000","last_clean":"2024-01-21T22:52:28.462701+0000","last_became_active":"2024-01-21T22:25:03.722758+0000","last_became_peered":"2024-01-21T22:25:03.722758+0000","last_unstale":"2024-01-21T22:52:28.462701+0000","last_undegraded":"2024-01-21T22:52:28.462701+0000","last_fullsized":"2024-01-21T22:52:28.462701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:16:40.192808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"25'224","reported_seq":264,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.418713+0000","last_change":"2024-01-21T22:25:04.844030+0000","last_active":"2024-01-21T22:52:41.418713+0000","last_peered":"2024-01-21T22:52:41.418713+0000","last_clean":"2024-01-21T22:52:41.418713+0000","last_became_active":"2024-01-21T22:25:04.843714+0000","last_became_peered":"2024-01-21T22:25:04.843714+0000","last_unstale":"2024-01-21T22:52:41.418713+0000","last_undegraded":"2024-01-21T22:52:41.418713+0000","last_fullsized":"2024-01-21T22:52:41.418713+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":174,"num_write_kb":5546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"25'1195","reported_seq":1223,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.522247+0000","last_change":"2024-01-21T22:25:03.702767+0000","last_active":"2024-01-21T22:52:28.522247+0000","last_peered":"2024-01-21T22:52:28.522247+0000","last_clean":"2024-01-21T22:52:28.522247+0000","last_became_active":"2024-01-21T22:25:03.702140+0000","last_became_peered":"2024-01-21T22:25:03.702140+0000","last_unstale":"2024-01-21T22:52:28.522247+0000","last_undegraded":"2024-01-21T22:52:28.522247+0000","last_fullsized":"2024-01-21T22:52:28.522247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1195,"log_dups_size":0,"ondisk_log_size":1195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1190,"num_write_kb":8787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"25'199","reported_seq":238,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.446059+0000","last_change":"2024-01-21T22:25:04.867173+0000","last_active":"2024-01-21T22:52:41.446059+0000","last_peered":"2024-01-21T22:52:41.446059+0000","last_clean":"2024-01-21T22:52:41.446059+0000","last_became_active":"2024-01-21T22:25:04.866945+0000","last_became_peered":"2024-01-21T22:25:04.866945+0000","last_unstale":"2024-01-21T22:52:41.446059+0000","last_undegraded":"2024-01-21T22:52:41.446059+0000","last_fullsized":"2024-01-21T22:52:41.446059+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":136,"num_write_kb":8996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:49.897688+0000","last_change":"2024-01-21T22:52:49.897754+0000","last_active":"2024-01-21T22:52:49.897688+0000","last_peered":"2024-01-21T22:52:49.897688+0000","last_clean":"2024-01-21T22:52:49.897688+0000","last_became_active":"2024-01-21T22:25:03.714039+0000","last_became_peered":"2024-01-21T22:25:03.714039+0000","last_unstale":"2024-01-21T22:52:49.897688+0000","last_undegraded":"2024-01-21T22:52:49.897688+0000","last_fullsized":"2024-01-21T22:52:49.897688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'3","last_scrub_stamp":"2024-01-21T22:52:49.897484+0000","last_deep_scrub":"25'3","last_deep_scrub_stamp":"2024-01-21T22:52:49.897484+0000","last_clean_scrub_stamp":"2024-01-21T22:52:49.897484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T10:44:49.265288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"25'210","reported_seq":249,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.369762+0000","last_change":"2024-01-21T22:25:04.886534+0000","last_active":"2024-01-21T22:52:41.369762+0000","last_peered":"2024-01-21T22:52:41.369762+0000","last_clean":"2024-01-21T22:52:41.369762+0000","last_became_active":"2024-01-21T22:25:04.885963+0000","last_became_peered":"2024-01-21T22:25:04.885963+0000","last_unstale":"2024-01-21T22:52:41.369762+0000","last_undegraded":"2024-01-21T22:52:41.369762+0000","last_fullsized":"2024-01-21T22:52:41.369762+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":149,"num_write_kb":6741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"25'5","reported_seq":41,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:48.887640+0000","last_change":"2024-01-21T22:52:48.887744+0000","last_active":"2024-01-21T22:52:48.887640+0000","last_peered":"2024-01-21T22:52:48.887640+0000","last_clean":"2024-01-21T22:52:48.887640+0000","last_became_active":"2024-01-21T22:25:03.717937+0000","last_became_peered":"2024-01-21T22:25:03.717937+0000","last_unstale":"2024-01-21T22:52:48.887640+0000","last_undegraded":"2024-01-21T22:52:48.887640+0000","last_fullsized":"2024-01-21T22:52:48.887640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'5","last_scrub_stamp":"2024-01-21T22:52:48.887398+0000","last_deep_scrub":"25'5","last_deep_scrub_stamp":"2024-01-21T22:52:48.887398+0000","last_clean_scrub_stamp":"2024-01-21T22:52:48.887398+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:19:29.088565+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"25'195","reported_seq":232,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.418509+0000","last_change":"2024-01-21T22:25:04.882743+0000","last_active":"2024-01-21T22:52:41.418509+0000","last_peered":"2024-01-21T22:52:41.418509+0000","last_clean":"2024-01-21T22:52:41.418509+0000","last_became_active":"2024-01-21T22:25:04.882478+0000","last_became_peered":"2024-01-21T22:25:04.882478+0000","last_unstale":"2024-01-21T22:52:41.418509+0000","last_undegraded":"2024-01-21T22:52:41.418509+0000","last_fullsized":"2024-01-21T22:52:41.418509+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2540,"num_write":148,"num_write_kb":5415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"25'468","reported_seq":495,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.588185+0000","last_change":"2024-01-21T22:25:03.703672+0000","last_active":"2024-01-21T22:52:33.588185+0000","last_peered":"2024-01-21T22:52:33.588185+0000","last_clean":"2024-01-21T22:52:33.588185+0000","last_became_active":"2024-01-21T22:25:03.703426+0000","last_became_peered":"2024-01-21T22:25:03.703426+0000","last_unstale":"2024-01-21T22:52:33.588185+0000","last_undegraded":"2024-01-21T22:52:33.588185+0000","last_fullsized":"2024-01-21T22:52:33.588185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:41:51.793206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":467,"num_write_kb":464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"25'215","reported_seq":264,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.415571+0000","last_change":"2024-01-21T22:25:04.887306+0000","last_active":"2024-01-21T22:52:41.415571+0000","last_peered":"2024-01-21T22:52:41.415571+0000","last_clean":"2024-01-21T22:52:41.415571+0000","last_became_active":"2024-01-21T22:25:04.886970+0000","last_became_peered":"2024-01-21T22:25:04.886970+0000","last_unstale":"2024-01-21T22:52:41.415571+0000","last_undegraded":"2024-01-21T22:52:41.415571+0000","last_fullsized":"2024-01-21T22:52:41.415571+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":5588,"num_write":173,"num_write_kb":7144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"25'882","reported_seq":918,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:50.132776+0000","last_change":"2024-01-21T22:52:50.132827+0000","last_active":"2024-01-21T22:52:50.132776+0000","last_peered":"2024-01-21T22:52:50.132776+0000","last_clean":"2024-01-21T22:52:50.132776+0000","last_became_active":"2024-01-21T22:25:03.713717+0000","last_became_peered":"2024-01-21T22:25:03.713717+0000","last_unstale":"2024-01-21T22:52:50.132776+0000","last_undegraded":"2024-01-21T22:52:50.132776+0000","last_fullsized":"2024-01-21T22:52:50.132776+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'882","last_scrub_stamp":"2024-01-21T22:52:50.132664+0000","last_deep_scrub":"25'882","last_deep_scrub_stamp":"2024-01-21T22:52:50.132664+0000","last_clean_scrub_stamp":"2024-01-21T22:52:50.132664+0000","objects_scrubbed":4,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:56:28.261981+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":877,"num_write_kb":4705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"25'169","reported_seq":214,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.426282+0000","last_change":"2024-01-21T22:25:04.881307+0000","last_active":"2024-01-21T22:52:41.426282+0000","last_peered":"2024-01-21T22:52:41.426282+0000","last_clean":"2024-01-21T22:52:41.426282+0000","last_became_active":"2024-01-21T22:25:04.880472+0000","last_became_peered":"2024-01-21T22:25:04.880472+0000","last_unstale":"2024-01-21T22:52:41.426282+0000","last_undegraded":"2024-01-21T22:52:41.426282+0000","last_fullsized":"2024-01-21T22:52:41.426282+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":4572,"num_write":122,"num_write_kb":6796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"25'1290","reported_seq":1317,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.495049+0000","last_change":"2024-01-21T22:25:03.721825+0000","last_active":"2024-01-21T22:52:33.495049+0000","last_peered":"2024-01-21T22:52:33.495049+0000","last_clean":"2024-01-21T22:52:33.495049+0000","last_became_active":"2024-01-21T22:25:03.721502+0000","last_became_peered":"2024-01-21T22:25:03.721502+0000","last_unstale":"2024-01-21T22:52:33.495049+0000","last_undegraded":"2024-01-21T22:52:33.495049+0000","last_fullsized":"2024-01-21T22:52:33.495049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1290,"log_dups_size":0,"ondisk_log_size":1290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:09:09.988399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1286,"num_write_kb":8849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"25'164","reported_seq":195,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.350536+0000","last_change":"2024-01-21T22:25:04.877455+0000","last_active":"2024-01-21T22:52:41.350536+0000","last_peered":"2024-01-21T22:52:41.350536+0000","last_clean":"2024-01-21T22:52:41.350536+0000","last_became_active":"2024-01-21T22:25:04.877257+0000","last_became_peered":"2024-01-21T22:25:04.877257+0000","last_unstale":"2024-01-21T22:52:41.350536+0000","last_undegraded":"2024-01-21T22:52:41.350536+0000","last_fullsized":"2024-01-21T22:52:41.350536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:34:14.550267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1270,"num_write":117,"num_write_kb":4465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"25'5","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:39.514340+0000","last_change":"2024-01-21T22:25:03.706077+0000","last_active":"2024-01-21T22:52:39.514340+0000","last_peered":"2024-01-21T22:52:39.514340+0000","last_clean":"2024-01-21T22:52:39.514340+0000","last_became_active":"2024-01-21T22:25:03.702798+0000","last_became_peered":"2024-01-21T22:25:03.702798+0000","last_unstale":"2024-01-21T22:52:39.514340+0000","last_undegraded":"2024-01-21T22:52:39.514340+0000","last_fullsized":"2024-01-21T22:52:39.514340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"25'231","reported_seq":267,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.416788+0000","last_change":"2024-01-21T22:25:04.890636+0000","last_active":"2024-01-21T22:52:41.416788+0000","last_peered":"2024-01-21T22:52:41.416788+0000","last_clean":"2024-01-21T22:52:41.416788+0000","last_became_active":"2024-01-21T22:25:04.890271+0000","last_became_peered":"2024-01-21T22:25:04.890271+0000","last_unstale":"2024-01-21T22:52:41.416788+0000","last_undegraded":"2024-01-21T22:52:41.416788+0000","last_fullsized":"2024-01-21T22:52:41.416788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":174,"num_write_kb":31551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"25'4","reported_seq":40,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:47.913534+0000","last_change":"2024-01-21T22:52:47.913624+0000","last_active":"2024-01-21T22:52:47.913534+0000","last_peered":"2024-01-21T22:52:47.913534+0000","last_clean":"2024-01-21T22:52:47.913534+0000","last_became_active":"2024-01-21T22:25:03.716846+0000","last_became_peered":"2024-01-21T22:25:03.716846+0000","last_unstale":"2024-01-21T22:52:47.913534+0000","last_undegraded":"2024-01-21T22:52:47.913534+0000","last_fullsized":"2024-01-21T22:52:47.913534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'4","last_scrub_stamp":"2024-01-21T22:52:47.913345+0000","last_deep_scrub":"25'4","last_deep_scrub_stamp":"2024-01-21T22:52:47.913345+0000","last_clean_scrub_stamp":"2024-01-21T22:52:47.913345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:28:57.508230+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"25'169","reported_seq":211,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:51.897255+0000","last_change":"2024-01-21T22:52:51.897302+0000","last_active":"2024-01-21T22:52:51.897255+0000","last_peered":"2024-01-21T22:52:51.897255+0000","last_clean":"2024-01-21T22:52:51.897255+0000","last_became_active":"2024-01-21T22:25:04.713566+0000","last_became_peered":"2024-01-21T22:25:04.713566+0000","last_unstale":"2024-01-21T22:52:51.897255+0000","last_undegraded":"2024-01-21T22:52:51.897255+0000","last_fullsized":"2024-01-21T22:52:51.897255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'169","last_scrub_stamp":"2024-01-21T22:52:51.897088+0000","last_deep_scrub":"25'169","last_deep_scrub_stamp":"2024-01-21T22:52:51.897088+0000","last_clean_scrub_stamp":"2024-01-21T22:52:51.897088+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T10:27:33.467202+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":119,"num_write_kb":3558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"25'14","reported_seq":50,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:53.234047+0000","last_change":"2024-01-21T22:52:53.234100+0000","last_active":"2024-01-21T22:52:53.234047+0000","last_peered":"2024-01-21T22:52:53.234047+0000","last_clean":"2024-01-21T22:52:53.234047+0000","last_became_active":"2024-01-21T22:25:03.702742+0000","last_became_peered":"2024-01-21T22:25:03.702742+0000","last_unstale":"2024-01-21T22:52:53.234047+0000","last_undegraded":"2024-01-21T22:52:53.234047+0000","last_fullsized":"2024-01-21T22:52:53.234047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'14","last_scrub_stamp":"2024-01-21T22:52:53.233961+0000","last_deep_scrub":"25'14","last_deep_scrub_stamp":"2024-01-21T22:52:53.233961+0000","last_clean_scrub_stamp":"2024-01-21T22:52:53.233961+0000","objects_scrubbed":4,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:01:43.450112+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"25'196","reported_seq":238,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:51.598026+0000","last_change":"2024-01-21T22:52:51.598050+0000","last_active":"2024-01-21T22:52:51.598026+0000","last_peered":"2024-01-21T22:52:51.598026+0000","last_clean":"2024-01-21T22:52:51.598026+0000","last_became_active":"2024-01-21T22:25:04.876169+0000","last_became_peered":"2024-01-21T22:25:04.876169+0000","last_unstale":"2024-01-21T22:52:51.598026+0000","last_undegraded":"2024-01-21T22:52:51.598026+0000","last_fullsized":"2024-01-21T22:52:51.598026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'196","last_scrub_stamp":"2024-01-21T22:52:51.597942+0000","last_deep_scrub":"25'196","last_deep_scrub_stamp":"2024-01-21T22:52:51.597942+0000","last_clean_scrub_stamp":"2024-01-21T22:52:51.597942+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:04:02.551018+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":142,"num_write_kb":8431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"25'8","reported_seq":45,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:52.936783+0000","last_change":"2024-01-21T22:52:52.936813+0000","last_active":"2024-01-21T22:52:52.936783+0000","last_peered":"2024-01-21T22:52:52.936783+0000","last_clean":"2024-01-21T22:52:52.936783+0000","last_became_active":"2024-01-21T22:25:03.720009+0000","last_became_peered":"2024-01-21T22:25:03.720009+0000","last_unstale":"2024-01-21T22:52:52.936783+0000","last_undegraded":"2024-01-21T22:52:52.936783+0000","last_fullsized":"2024-01-21T22:52:52.936783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'8","last_scrub_stamp":"2024-01-21T22:52:52.936704+0000","last_deep_scrub":"25'8","last_deep_scrub_stamp":"2024-01-21T22:52:52.936704+0000","last_clean_scrub_stamp":"2024-01-21T22:52:52.936704+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:13:53.262696+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"25'153","reported_seq":202,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.368548+0000","last_change":"2024-01-21T22:25:04.883112+0000","last_active":"2024-01-21T22:52:41.368548+0000","last_peered":"2024-01-21T22:52:41.368548+0000","last_clean":"2024-01-21T22:52:41.368548+0000","last_became_active":"2024-01-21T22:25:04.882767+0000","last_became_peered":"2024-01-21T22:25:04.882767+0000","last_unstale":"2024-01-21T22:52:41.368548+0000","last_undegraded":"2024-01-21T22:52:41.368548+0000","last_fullsized":"2024-01-21T22:52:41.368548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":5334,"num_write":115,"num_write_kb":10137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"25'300","reported_seq":329,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.539292+0000","last_change":"2024-01-21T22:25:03.703516+0000","last_active":"2024-01-21T22:52:33.539292+0000","last_peered":"2024-01-21T22:52:33.539292+0000","last_clean":"2024-01-21T22:52:33.539292+0000","last_became_active":"2024-01-21T22:25:03.703256+0000","last_became_peered":"2024-01-21T22:25:03.703256+0000","last_unstale":"2024-01-21T22:52:33.539292+0000","last_undegraded":"2024-01-21T22:52:33.539292+0000","last_fullsized":"2024-01-21T22:52:33.539292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":296,"num_write_kb":4241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"25'166","reported_seq":200,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.341937+0000","last_change":"2024-01-21T22:25:38.276745+0000","last_active":"2024-01-21T22:52:41.341937+0000","last_peered":"2024-01-21T22:52:41.341937+0000","last_clean":"2024-01-21T22:52:41.341937+0000","last_became_active":"2024-01-21T22:25:38.276512+0000","last_became_peered":"2024-01-21T22:25:38.276512+0000","last_unstale":"2024-01-21T22:52:41.341937+0000","last_undegraded":"2024-01-21T22:52:41.341937+0000","last_fullsized":"2024-01-21T22:52:41.341937+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:23:12.153042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":119,"num_write_kb":11355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"25'627","reported_seq":656,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:33.543833+0000","last_change":"2024-01-21T22:25:03.708366+0000","last_active":"2024-01-21T22:52:33.543833+0000","last_peered":"2024-01-21T22:52:33.543833+0000","last_clean":"2024-01-21T22:52:33.543833+0000","last_became_active":"2024-01-21T22:25:03.708101+0000","last_became_peered":"2024-01-21T22:25:03.708101+0000","last_unstale":"2024-01-21T22:52:33.543833+0000","last_undegraded":"2024-01-21T22:52:33.543833+0000","last_fullsized":"2024-01-21T22:52:33.543833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":625,"num_write_kb":4435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"25'198","reported_seq":233,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.428783+0000","last_change":"2024-01-21T22:25:04.877993+0000","last_active":"2024-01-21T22:52:41.428783+0000","last_peered":"2024-01-21T22:52:41.428783+0000","last_clean":"2024-01-21T22:52:41.428783+0000","last_became_active":"2024-01-21T22:25:04.877661+0000","last_became_peered":"2024-01-21T22:25:04.877661+0000","last_unstale":"2024-01-21T22:52:41.428783+0000","last_undegraded":"2024-01-21T22:52:41.428783+0000","last_fullsized":"2024-01-21T22:52:41.428783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":145,"num_write_kb":7036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"25'2","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.462672+0000","last_change":"2024-01-21T22:25:03.721559+0000","last_active":"2024-01-21T22:52:28.462672+0000","last_peered":"2024-01-21T22:52:28.462672+0000","last_clean":"2024-01-21T22:52:28.462672+0000","last_became_active":"2024-01-21T22:25:03.721281+0000","last_became_peered":"2024-01-21T22:25:03.721281+0000","last_unstale":"2024-01-21T22:52:28.462672+0000","last_undegraded":"2024-01-21T22:52:28.462672+0000","last_fullsized":"2024-01-21T22:52:28.462672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:55:19.679388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"25'173","reported_seq":218,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.410968+0000","last_change":"2024-01-21T22:25:04.883087+0000","last_active":"2024-01-21T22:52:41.410968+0000","last_peered":"2024-01-21T22:52:41.410968+0000","last_clean":"2024-01-21T22:52:41.410968+0000","last_became_active":"2024-01-21T22:25:04.880227+0000","last_became_peered":"2024-01-21T22:25:04.880227+0000","last_unstale":"2024-01-21T22:52:41.410968+0000","last_undegraded":"2024-01-21T22:52:41.410968+0000","last_fullsized":"2024-01-21T22:52:41.410968+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":4318,"num_write":118,"num_write_kb":4353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"25'119","reported_seq":193,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:48.910070+0000","last_change":"2024-01-21T22:52:48.910070+0000","last_active":"2024-01-21T22:52:48.910070+0000","last_peered":"2024-01-21T22:52:48.910070+0000","last_clean":"2024-01-21T22:52:48.910070+0000","last_became_active":"2024-01-21T22:24:52.967018+0000","last_became_peered":"2024-01-21T22:24:52.967018+0000","last_unstale":"2024-01-21T22:52:48.910070+0000","last_undegraded":"2024-01-21T22:52:48.910070+0000","last_fullsized":"2024-01-21T22:52:48.910070+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'119","last_scrub_stamp":"2024-01-21T22:52:48.909687+0000","last_deep_scrub":"25'119","last_deep_scrub_stamp":"2024-01-21T22:52:48.909687+0000","last_clean_scrub_stamp":"2024-01-21T22:52:48.909687+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:14:14.043092+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"25'6","reported_seq":36,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:39.512718+0000","last_change":"2024-01-21T22:25:03.707407+0000","last_active":"2024-01-21T22:52:39.512718+0000","last_peered":"2024-01-21T22:52:39.512718+0000","last_clean":"2024-01-21T22:52:39.512718+0000","last_became_active":"2024-01-21T22:25:03.707085+0000","last_became_peered":"2024-01-21T22:25:03.707085+0000","last_unstale":"2024-01-21T22:52:39.512718+0000","last_undegraded":"2024-01-21T22:52:39.512718+0000","last_fullsized":"2024-01-21T22:52:39.512718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"25'177","reported_seq":216,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.444669+0000","last_change":"2024-01-21T22:25:04.868564+0000","last_active":"2024-01-21T22:52:41.444669+0000","last_peered":"2024-01-21T22:52:41.444669+0000","last_clean":"2024-01-21T22:52:41.444669+0000","last_became_active":"2024-01-21T22:25:04.868247+0000","last_became_peered":"2024-01-21T22:25:04.868247+0000","last_unstale":"2024-01-21T22:52:41.444669+0000","last_undegraded":"2024-01-21T22:52:41.444669+0000","last_fullsized":"2024-01-21T22:52:41.444669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2540,"num_write":132,"num_write_kb":4885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"25'1293","reported_seq":1323,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:39.514713+0000","last_change":"2024-01-21T22:25:03.706070+0000","last_active":"2024-01-21T22:52:39.514713+0000","last_peered":"2024-01-21T22:52:39.514713+0000","last_clean":"2024-01-21T22:52:39.514713+0000","last_became_active":"2024-01-21T22:25:03.705785+0000","last_became_peered":"2024-01-21T22:25:03.705785+0000","last_unstale":"2024-01-21T22:52:39.514713+0000","last_undegraded":"2024-01-21T22:52:39.514713+0000","last_fullsized":"2024-01-21T22:52:39.514713+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1293,"log_dups_size":0,"ondisk_log_size":1293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1286,"num_write_kb":8869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"25'188","reported_seq":228,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.348728+0000","last_change":"2024-01-21T22:25:04.845436+0000","last_active":"2024-01-21T22:52:41.348728+0000","last_peered":"2024-01-21T22:52:41.348728+0000","last_clean":"2024-01-21T22:52:41.348728+0000","last_became_active":"2024-01-21T22:25:04.845274+0000","last_became_peered":"2024-01-21T22:25:04.845274+0000","last_unstale":"2024-01-21T22:52:41.348728+0000","last_undegraded":"2024-01-21T22:52:41.348728+0000","last_fullsized":"2024-01-21T22:52:41.348728+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:08:36.686787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":151,"num_write_kb":10622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"25'756","reported_seq":1531,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:39.468646+0000","last_change":"2024-01-21T22:25:03.709174+0000","last_active":"2024-01-21T22:52:39.468646+0000","last_peered":"2024-01-21T22:52:39.468646+0000","last_clean":"2024-01-21T22:52:39.468646+0000","last_became_active":"2024-01-21T22:25:03.708894+0000","last_became_peered":"2024-01-21T22:25:03.708894+0000","last_unstale":"2024-01-21T22:52:39.468646+0000","last_undegraded":"2024-01-21T22:52:39.468646+0000","last_fullsized":"2024-01-21T22:52:39.468646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:19:26.591140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":610747,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":746,"num_read_kb":1116,"num_write":750,"num_write_kb":1119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"25'188","reported_seq":231,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.418771+0000","last_change":"2024-01-21T22:25:04.711947+0000","last_active":"2024-01-21T22:52:41.418771+0000","last_peered":"2024-01-21T22:52:41.418771+0000","last_clean":"2024-01-21T22:52:41.418771+0000","last_became_active":"2024-01-21T22:25:04.711694+0000","last_became_peered":"2024-01-21T22:25:04.711694+0000","last_unstale":"2024-01-21T22:52:41.418771+0000","last_undegraded":"2024-01-21T22:52:41.418771+0000","last_fullsized":"2024-01-21T22:52:41.418771+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3556,"num_write":135,"num_write_kb":5944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"25'1296","reported_seq":1326,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:39.514984+0000","last_change":"2024-01-21T22:25:03.706355+0000","last_active":"2024-01-21T22:52:39.514984+0000","last_peered":"2024-01-21T22:52:39.514984+0000","last_clean":"2024-01-21T22:52:39.514984+0000","last_became_active":"2024-01-21T22:25:03.703876+0000","last_became_peered":"2024-01-21T22:25:03.703876+0000","last_unstale":"2024-01-21T22:52:39.514984+0000","last_undegraded":"2024-01-21T22:52:39.514984+0000","last_fullsized":"2024-01-21T22:52:39.514984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1296,"log_dups_size":0,"ondisk_log_size":1296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1291,"num_write_kb":8882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"25'230","reported_seq":268,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.423298+0000","last_change":"2024-01-21T22:25:04.885997+0000","last_active":"2024-01-21T22:52:41.423298+0000","last_peered":"2024-01-21T22:52:41.423298+0000","last_clean":"2024-01-21T22:52:41.423298+0000","last_became_active":"2024-01-21T22:25:04.884162+0000","last_became_peered":"2024-01-21T22:25:04.884162+0000","last_unstale":"2024-01-21T22:52:41.423298+0000","last_undegraded":"2024-01-21T22:52:41.423298+0000","last_fullsized":"2024-01-21T22:52:41.423298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":180,"num_write_kb":15842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"25'4","reported_seq":34,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:39.512609+0000","last_change":"2024-01-21T22:25:03.708300+0000","last_active":"2024-01-21T22:52:39.512609+0000","last_peered":"2024-01-21T22:52:39.512609+0000","last_clean":"2024-01-21T22:52:39.512609+0000","last_became_active":"2024-01-21T22:25:03.708036+0000","last_became_peered":"2024-01-21T22:25:03.708036+0000","last_unstale":"2024-01-21T22:52:39.512609+0000","last_undegraded":"2024-01-21T22:52:39.512609+0000","last_fullsized":"2024-01-21T22:52:39.512609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"25'188","reported_seq":241,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:52.078481+0000","last_change":"2024-01-21T22:52:52.078542+0000","last_active":"2024-01-21T22:52:52.078481+0000","last_peered":"2024-01-21T22:52:52.078481+0000","last_clean":"2024-01-21T22:52:52.078481+0000","last_became_active":"2024-01-21T22:25:04.844008+0000","last_became_peered":"2024-01-21T22:25:04.844008+0000","last_unstale":"2024-01-21T22:52:52.078481+0000","last_undegraded":"2024-01-21T22:52:52.078481+0000","last_fullsized":"2024-01-21T22:52:52.078481+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'188","last_scrub_stamp":"2024-01-21T22:52:52.078302+0000","last_deep_scrub":"25'188","last_deep_scrub_stamp":"2024-01-21T22:52:52.078302+0000","last_clean_scrub_stamp":"2024-01-21T22:52:52.078302+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:05:20.309821+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":4064,"num_write":142,"num_write_kb":4003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"25'3","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:39.514068+0000","last_change":"2024-01-21T22:25:03.699093+0000","last_active":"2024-01-21T22:52:39.514068+0000","last_peered":"2024-01-21T22:52:39.514068+0000","last_clean":"2024-01-21T22:52:39.514068+0000","last_became_active":"2024-01-21T22:25:03.698443+0000","last_became_peered":"2024-01-21T22:25:03.698443+0000","last_unstale":"2024-01-21T22:52:39.514068+0000","last_undegraded":"2024-01-21T22:52:39.514068+0000","last_fullsized":"2024-01-21T22:52:39.514068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"25'213","reported_seq":248,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.343518+0000","last_change":"2024-01-21T22:25:04.842733+0000","last_active":"2024-01-21T22:52:41.343518+0000","last_peered":"2024-01-21T22:52:41.343518+0000","last_clean":"2024-01-21T22:52:41.343518+0000","last_became_active":"2024-01-21T22:25:04.842479+0000","last_became_peered":"2024-01-21T22:25:04.842479+0000","last_unstale":"2024-01-21T22:52:41.343518+0000","last_undegraded":"2024-01-21T22:52:41.343518+0000","last_fullsized":"2024-01-21T22:52:41.343518+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:51:06.931790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":156,"num_write_kb":5205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"25'1421","reported_seq":1450,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.439293+0000","last_change":"2024-01-21T22:25:03.720306+0000","last_active":"2024-01-21T22:52:41.439293+0000","last_peered":"2024-01-21T22:52:41.439293+0000","last_clean":"2024-01-21T22:52:41.439293+0000","last_became_active":"2024-01-21T22:25:03.720067+0000","last_became_peered":"2024-01-21T22:25:03.720067+0000","last_unstale":"2024-01-21T22:52:41.439293+0000","last_undegraded":"2024-01-21T22:52:41.439293+0000","last_fullsized":"2024-01-21T22:52:41.439293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":1421,"log_dups_size":0,"ondisk_log_size":1421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3488,"num_write_kb":1512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"25'225","reported_seq":276,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.444498+0000","last_change":"2024-01-21T22:25:04.859628+0000","last_active":"2024-01-21T22:52:41.444498+0000","last_peered":"2024-01-21T22:52:41.444498+0000","last_clean":"2024-01-21T22:52:41.444498+0000","last_became_active":"2024-01-21T22:25:04.859270+0000","last_became_peered":"2024-01-21T22:25:04.859270+0000","last_unstale":"2024-01-21T22:52:41.444498+0000","last_undegraded":"2024-01-21T22:52:41.444498+0000","last_fullsized":"2024-01-21T22:52:41.444498+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":5842,"num_write":177,"num_write_kb":7531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"25'7","reported_seq":36,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:39.514449+0000","last_change":"2024-01-21T22:25:03.722006+0000","last_active":"2024-01-21T22:52:39.514449+0000","last_peered":"2024-01-21T22:52:39.514449+0000","last_clean":"2024-01-21T22:52:39.514449+0000","last_became_active":"2024-01-21T22:25:03.721275+0000","last_became_peered":"2024-01-21T22:25:03.721275+0000","last_unstale":"2024-01-21T22:52:39.514449+0000","last_undegraded":"2024-01-21T22:52:39.514449+0000","last_fullsized":"2024-01-21T22:52:39.514449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"25'182","reported_seq":223,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.445566+0000","last_change":"2024-01-21T22:25:04.858864+0000","last_active":"2024-01-21T22:52:41.445566+0000","last_peered":"2024-01-21T22:52:41.445566+0000","last_clean":"2024-01-21T22:52:41.445566+0000","last_became_active":"2024-01-21T22:25:04.858586+0000","last_became_peered":"2024-01-21T22:25:04.858586+0000","last_unstale":"2024-01-21T22:52:41.445566+0000","last_undegraded":"2024-01-21T22:52:41.445566+0000","last_fullsized":"2024-01-21T22:52:41.445566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":132,"num_write_kb":7745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"25'672","reported_seq":709,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:49.868698+0000","last_change":"2024-01-21T22:52:49.868747+0000","last_active":"2024-01-21T22:52:49.868698+0000","last_peered":"2024-01-21T22:52:49.868698+0000","last_clean":"2024-01-21T22:52:49.868698+0000","last_became_active":"2024-01-21T22:25:03.718171+0000","last_became_peered":"2024-01-21T22:25:03.718171+0000","last_unstale":"2024-01-21T22:52:49.868698+0000","last_undegraded":"2024-01-21T22:52:49.868698+0000","last_fullsized":"2024-01-21T22:52:49.868698+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'672","last_scrub_stamp":"2024-01-21T22:52:49.868570+0000","last_deep_scrub":"25'672","last_deep_scrub_stamp":"2024-01-21T22:52:49.868570+0000","last_clean_scrub_stamp":"2024-01-21T22:52:49.868570+0000","objects_scrubbed":1,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:50:05.726919+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":666,"num_write_kb":4450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"25'192","reported_seq":237,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:52.570424+0000","last_change":"2024-01-21T22:52:52.570449+0000","last_active":"2024-01-21T22:52:52.570424+0000","last_peered":"2024-01-21T22:52:52.570424+0000","last_clean":"2024-01-21T22:52:52.570424+0000","last_became_active":"2024-01-21T22:25:04.875368+0000","last_became_peered":"2024-01-21T22:25:04.875368+0000","last_unstale":"2024-01-21T22:52:52.570424+0000","last_undegraded":"2024-01-21T22:52:52.570424+0000","last_fullsized":"2024-01-21T22:52:52.570424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'192","last_scrub_stamp":"2024-01-21T22:52:52.570345+0000","last_deep_scrub":"25'192","last_deep_scrub_stamp":"2024-01-21T22:52:52.570345+0000","last_clean_scrub_stamp":"2024-01-21T22:52:52.570345+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:14:45.162994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":138,"num_write_kb":6957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"25'4","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:39.497119+0000","last_change":"2024-01-21T22:25:03.722820+0000","last_active":"2024-01-21T22:52:39.497119+0000","last_peered":"2024-01-21T22:52:39.497119+0000","last_clean":"2024-01-21T22:52:39.497119+0000","last_became_active":"2024-01-21T22:25:03.722525+0000","last_became_peered":"2024-01-21T22:25:03.722525+0000","last_unstale":"2024-01-21T22:52:39.497119+0000","last_undegraded":"2024-01-21T22:52:39.497119+0000","last_fullsized":"2024-01-21T22:52:39.497119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"25'178","reported_seq":219,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.445090+0000","last_change":"2024-01-21T22:25:04.857451+0000","last_active":"2024-01-21T22:52:41.445090+0000","last_peered":"2024-01-21T22:52:41.445090+0000","last_clean":"2024-01-21T22:52:41.445090+0000","last_became_active":"2024-01-21T22:25:04.857205+0000","last_became_peered":"2024-01-21T22:25:04.857205+0000","last_unstale":"2024-01-21T22:52:41.445090+0000","last_undegraded":"2024-01-21T22:52:41.445090+0000","last_fullsized":"2024-01-21T22:52:41.445090+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":126,"num_write_kb":4840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"25'2","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:51:08.155593+0000","last_change":"2024-01-21T22:25:03.708574+0000","last_active":"2024-01-21T22:51:08.155593+0000","last_peered":"2024-01-21T22:51:08.155593+0000","last_clean":"2024-01-21T22:51:08.155593+0000","last_became_active":"2024-01-21T22:25:03.708318+0000","last_became_peered":"2024-01-21T22:25:03.708318+0000","last_unstale":"2024-01-21T22:51:08.155593+0000","last_undegraded":"2024-01-21T22:51:08.155593+0000","last_fullsized":"2024-01-21T22:51:08.155593+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:50:40.058463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"25'222","reported_seq":282,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.330361+0000","last_change":"2024-01-21T22:25:04.843079+0000","last_active":"2024-01-21T22:52:41.330361+0000","last_peered":"2024-01-21T22:52:41.330361+0000","last_clean":"2024-01-21T22:52:41.330361+0000","last_became_active":"2024-01-21T22:25:04.842815+0000","last_became_peered":"2024-01-21T22:25:04.842815+0000","last_unstale":"2024-01-21T22:52:41.330361+0000","last_undegraded":"2024-01-21T22:52:41.330361+0000","last_fullsized":"2024-01-21T22:52:41.330361+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:16:41.440927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":8382,"num_write":174,"num_write_kb":9170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"25'764","reported_seq":793,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:39.514103+0000","last_change":"2024-01-21T22:25:03.710506+0000","last_active":"2024-01-21T22:52:39.514103+0000","last_peered":"2024-01-21T22:52:39.514103+0000","last_clean":"2024-01-21T22:52:39.514103+0000","last_became_active":"2024-01-21T22:25:03.710262+0000","last_became_peered":"2024-01-21T22:25:03.710262+0000","last_unstale":"2024-01-21T22:52:39.514103+0000","last_undegraded":"2024-01-21T22:52:39.514103+0000","last_fullsized":"2024-01-21T22:52:39.514103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":764,"log_dups_size":0,"ondisk_log_size":764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":756,"num_write_kb":4574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"25'193","reported_seq":243,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:50.594522+0000","last_change":"2024-01-21T22:52:50.594559+0000","last_active":"2024-01-21T22:52:50.594522+0000","last_peered":"2024-01-21T22:52:50.594522+0000","last_clean":"2024-01-21T22:52:50.594522+0000","last_became_active":"2024-01-21T22:25:04.875619+0000","last_became_peered":"2024-01-21T22:25:04.875619+0000","last_unstale":"2024-01-21T22:52:50.594522+0000","last_undegraded":"2024-01-21T22:52:50.594522+0000","last_fullsized":"2024-01-21T22:52:50.594522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'193","last_scrub_stamp":"2024-01-21T22:52:50.594392+0000","last_deep_scrub":"25'193","last_deep_scrub_stamp":"2024-01-21T22:52:50.594392+0000","last_clean_scrub_stamp":"2024-01-21T22:52:50.594392+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:18:57.135293+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3556,"num_write":140,"num_write_kb":4746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"25'155","reported_seq":189,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.406999+0000","last_change":"2024-01-21T22:25:04.891067+0000","last_active":"2024-01-21T22:52:41.406999+0000","last_peered":"2024-01-21T22:52:41.406999+0000","last_clean":"2024-01-21T22:52:41.406999+0000","last_became_active":"2024-01-21T22:25:04.890515+0000","last_became_peered":"2024-01-21T22:25:04.890515+0000","last_unstale":"2024-01-21T22:52:41.406999+0000","last_undegraded":"2024-01-21T22:52:41.406999+0000","last_fullsized":"2024-01-21T22:52:41.406999+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":88,"num_write_kb":3797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"25'206","reported_seq":242,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:51.899411+0000","last_change":"2024-01-21T22:52:51.899452+0000","last_active":"2024-01-21T22:52:51.899411+0000","last_peered":"2024-01-21T22:52:51.899411+0000","last_clean":"2024-01-21T22:52:51.899411+0000","last_became_active":"2024-01-21T22:25:03.718163+0000","last_became_peered":"2024-01-21T22:25:03.718163+0000","last_unstale":"2024-01-21T22:52:51.899411+0000","last_undegraded":"2024-01-21T22:52:51.899411+0000","last_fullsized":"2024-01-21T22:52:51.899411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'206","last_scrub_stamp":"2024-01-21T22:52:51.899295+0000","last_deep_scrub":"25'206","last_deep_scrub_stamp":"2024-01-21T22:52:51.899295+0000","last_clean_scrub_stamp":"2024-01-21T22:52:51.899295+0000","objects_scrubbed":2,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T10:51:29.160818+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1916505,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":203,"num_write_kb":1995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"25'212","reported_seq":245,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.446302+0000","last_change":"2024-01-21T22:25:04.889166+0000","last_active":"2024-01-21T22:52:41.446302+0000","last_peered":"2024-01-21T22:52:41.446302+0000","last_clean":"2024-01-21T22:52:41.446302+0000","last_became_active":"2024-01-21T22:25:04.888910+0000","last_became_peered":"2024-01-21T22:25:04.888910+0000","last_unstale":"2024-01-21T22:52:41.446302+0000","last_undegraded":"2024-01-21T22:52:41.446302+0000","last_fullsized":"2024-01-21T22:52:41.446302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":155,"num_write_kb":13062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"25'1487","reported_seq":1523,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:50.946510+0000","last_change":"2024-01-21T22:52:50.946563+0000","last_active":"2024-01-21T22:52:50.946510+0000","last_peered":"2024-01-21T22:52:50.946510+0000","last_clean":"2024-01-21T22:52:50.946510+0000","last_became_active":"2024-01-21T22:25:03.718649+0000","last_became_peered":"2024-01-21T22:25:03.718649+0000","last_unstale":"2024-01-21T22:52:50.946510+0000","last_undegraded":"2024-01-21T22:52:50.946510+0000","last_fullsized":"2024-01-21T22:52:50.946510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1487","last_scrub_stamp":"2024-01-21T22:52:50.946367+0000","last_deep_scrub":"25'1487","last_deep_scrub_stamp":"2024-01-21T22:52:50.946367+0000","last_clean_scrub_stamp":"2024-01-21T22:52:50.946367+0000","objects_scrubbed":3,"log_size":1487,"log_dups_size":0,"ondisk_log_size":1487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:49:11.859146+0000","scrub_duration":64,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1478,"num_write_kb":9119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"25'210","reported_seq":266,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.419571+0000","last_change":"2024-01-21T22:25:04.715429+0000","last_active":"2024-01-21T22:52:41.419571+0000","last_peered":"2024-01-21T22:52:41.419571+0000","last_clean":"2024-01-21T22:52:41.419571+0000","last_became_active":"2024-01-21T22:25:04.714720+0000","last_became_peered":"2024-01-21T22:25:04.714720+0000","last_unstale":"2024-01-21T22:52:41.419571+0000","last_undegraded":"2024-01-21T22:52:41.419571+0000","last_fullsized":"2024-01-21T22:52:41.419571+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":6858,"num_write":147,"num_write_kb":17332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"25'747","reported_seq":777,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:28.458061+0000","last_change":"2024-01-21T22:25:03.703289+0000","last_active":"2024-01-21T22:52:28.458061+0000","last_peered":"2024-01-21T22:52:28.458061+0000","last_clean":"2024-01-21T22:52:28.458061+0000","last_became_active":"2024-01-21T22:25:03.703020+0000","last_became_peered":"2024-01-21T22:25:03.703020+0000","last_unstale":"2024-01-21T22:52:28.458061+0000","last_undegraded":"2024-01-21T22:52:28.458061+0000","last_fullsized":"2024-01-21T22:52:28.458061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:02.196506+0000","last_clean_scrub_stamp":"2024-01-21T22:25:02.196506+0000","objects_scrubbed":0,"log_size":747,"log_dups_size":0,"ondisk_log_size":747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":743,"num_write_kb":4567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"25'216","reported_seq":267,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:41.421253+0000","last_change":"2024-01-21T22:25:04.715894+0000","last_active":"2024-01-21T22:52:41.421253+0000","last_peered":"2024-01-21T22:52:41.421253+0000","last_clean":"2024-01-21T22:52:41.421253+0000","last_became_active":"2024-01-21T22:25:04.715652+0000","last_became_peered":"2024-01-21T22:25:04.715652+0000","last_unstale":"2024-01-21T22:52:41.421253+0000","last_undegraded":"2024-01-21T22:52:41.421253+0000","last_fullsized":"2024-01-21T22:52:41.421253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-01-21T22:25:03.444625+0000","last_clean_scrub_stamp":"2024-01-21T22:25:03.444625+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":5588,"num_write":182,"num_write_kb":13243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"25'765","reported_seq":803,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:52.281253+0000","last_change":"2024-01-21T22:52:52.281313+0000","last_active":"2024-01-21T22:52:52.281253+0000","last_peered":"2024-01-21T22:52:52.281253+0000","last_clean":"2024-01-21T22:52:52.281253+0000","last_became_active":"2024-01-21T22:25:03.702087+0000","last_became_peered":"2024-01-21T22:25:03.702087+0000","last_unstale":"2024-01-21T22:52:52.281253+0000","last_undegraded":"2024-01-21T22:52:52.281253+0000","last_fullsized":"2024-01-21T22:52:52.281253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'765","last_scrub_stamp":"2024-01-21T22:52:52.281140+0000","last_deep_scrub":"25'765","last_deep_scrub_stamp":"2024-01-21T22:52:52.281140+0000","last_clean_scrub_stamp":"2024-01-21T22:52:52.281140+0000","objects_scrubbed":3,"log_size":765,"log_dups_size":0,"ondisk_log_size":765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:37:56.693317+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":760,"num_write_kb":4560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":833,"num_read_kb":211582,"num_write":8969,"num_write_kb":506711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12284,"ondisk_log_size":12284,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":82220498,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":746,"num_read_kb":1116,"num_write":17695,"num_write_kb":90278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164503552,"data_stored":164440996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15769,"ondisk_log_size":15769,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542486,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62376,"kb_used_data":13152,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94309464,"statfs":{"total":96636764160,"available":96572891136,"internally_reserved":0,"allocated":13467648,"data_stored":13290724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66800000000000004,"15min":0.68300000000000005},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.872,"5min":1.0189999999999999,"15min":1.0640000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70599999999999996,"15min":0.69699999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.024,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.34799999999999998}]},{"osd":1,"last update":"Sun Jan 21 22:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.70299999999999996,"15min":0.70899999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.34000000000000002},"max":{"1min":0.98099999999999998,"5min":1.159,"15min":1.4179999999999999},"last":0.44},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.41299999999999998}]},{"osd":2,"last update":"Sun Jan 21 22:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.39300000000000002},"max":{"1min":1.002,"5min":1.0069999999999999,"15min":1.052},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.72199999999999998,"15min":0.72299999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.38},"max":{"1min":1.0409999999999999,"5min":1.1140000000000001,"15min":1.1799999999999999},"last":0.47099999999999997}]},{"osd":3,"last update":"Sun Jan 21 22:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69299999999999995,"15min":0.71199999999999997},"min":{"1min":0.45900000000000002,"5min":0.42699999999999999,"15min":0.42099999999999999},"max":{"1min":0.89900000000000002,"5min":1.093,"15min":1.1539999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95799999999999996,"5min":1.117,"15min":1.181},"last":0.503}]},{"osd":4,"last update":"Sun Jan 21 22:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35099999999999998,"15min":0.34499999999999997},"min":{"1min":0.25,"5min":0.245,"15min":0.22700000000000001},"max":{"1min":0.505,"5min":0.57099999999999995,"15min":0.58299999999999996},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.35699999999999998,"15min":0.34799999999999998},"min":{"1min":0.27300000000000002,"5min":0.248,"15min":0.23000000000000001},"max":{"1min":0.56499999999999995,"5min":0.66700000000000004,"15min":0.68100000000000005},"last":0.377}]},{"osd":5,"last update":"Sun Jan 21 22:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.34300000000000003},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.46999999999999997,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.373},"min":{"1min":0.29099999999999998,"5min":0.26700000000000002,"15min":0.249},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.624},"last":0.29599999999999999}]},{"osd":6,"last update":"Sun Jan 21 22:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40100000000000002,"15min":0.39600000000000002},"min":{"1min":0.32500000000000001,"5min":0.312,"15min":0.27800000000000002},"max":{"1min":0.55000000000000004,"5min":0.57199999999999995,"15min":0.67300000000000004},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.38800000000000001,"15min":0.38500000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.26400000000000001},"max":{"1min":0.55000000000000004,"5min":0.60699999999999998,"15min":0.63700000000000001},"last":0.42499999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542486,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43720,"kb_used_data":12544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328120,"statfs":{"total":96636764160,"available":96591994880,"internally_reserved":0,"allocated":12845056,"data_stored":12679853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69299999999999995,"15min":0.70099999999999996},"min":{"1min":0.497,"5min":0.48299999999999998,"15min":0.45600000000000002},"max":{"1min":0.89200000000000002,"5min":1.0189999999999999,"15min":3.3380000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.54100000000000004,"5min":0.495,"15min":0.44600000000000001},"max":{"1min":0.93799999999999994,"5min":1.038,"15min":1.0669999999999999},"last":0.39900000000000002}]},{"osd":1,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72099999999999997,"15min":0.70699999999999996},"min":{"1min":0.46500000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0529999999999999,"5min":1.0860000000000001,"15min":2.9860000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72299999999999998,"15min":0.72099999999999997},"min":{"1min":0.52900000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.1020000000000001},"last":0.48499999999999999}]},{"osd":2,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.49299999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.88200000000000001,"5min":1.1220000000000001,"15min":7.5439999999999996},"last":0.374},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71999999999999997,"15min":0.73199999999999998},"min":{"1min":0.59499999999999997,"5min":0.56200000000000006,"15min":0.495},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.40999999999999998}]},{"osd":3,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72399999999999998,"15min":0.70499999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.42599999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73099999999999998,"15min":0.78100000000000003},"min":{"1min":0.56399999999999995,"5min":0.52700000000000002,"15min":0.39400000000000002},"max":{"1min":0.90100000000000002,"5min":1.075,"15min":8.4689999999999994},"last":0.497}]},{"osd":4,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34300000000000003,"15min":0.33800000000000002},"min":{"1min":0.26000000000000001,"5min":0.245,"15min":0.222},"max":{"1min":0.45200000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.35599999999999998,"15min":0.35199999999999998},"min":{"1min":0.24199999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.45100000000000001,"5min":0.45600000000000002,"15min":0.91900000000000004},"last":0.32700000000000001}]},{"osd":5,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35199999999999998,"15min":0.35399999999999998},"min":{"1min":0.30199999999999999,"5min":0.26400000000000001,"15min":0.26100000000000001},"max":{"1min":0.40200000000000002,"5min":0.53800000000000003,"15min":0.61299999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.377,"15min":0.36799999999999999},"min":{"1min":0.29599999999999999,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.45300000000000001,"5min":0.46800000000000003,"15min":0.52200000000000002},"last":0.35199999999999998}]},{"osd":7,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.38},"min":{"1min":0.29799999999999999,"5min":0.27100000000000002,"15min":0.26600000000000001},"max":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.96099999999999997},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.39200000000000002,"15min":0.39100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28799999999999998,"15min":0.27300000000000002},"max":{"1min":0.45200000000000001,"5min":0.54900000000000004,"15min":0.98699999999999999},"last":0.46300000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542486,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":101612,"kb_used_data":26020,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94270228,"statfs":{"total":96636764160,"available":96532713472,"internally_reserved":0,"allocated":26644480,"data_stored":26463972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67700000000000005,"15min":0.68899999999999995},"min":{"1min":0.43099999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96999999999999997,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.69599999999999995,"15min":0.69899999999999995},"min":{"1min":0.47599999999999998,"5min":0.432,"15min":0.41499999999999998},"max":{"1min":0.97799999999999998,"5min":1.0149999999999999,"15min":1.1020000000000001},"last":0.439}]},{"osd":1,"last update":"Sun Jan 21 22:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68200000000000005,"15min":0.69099999999999995},"min":{"1min":0.42799999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":1.1950000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.72199999999999998,"15min":0.72499999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.40999999999999998},"max":{"1min":0.85999999999999999,"5min":1.111,"15min":1.111},"last":0.499}]},{"osd":2,"last update":"Sun Jan 21 22:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.70099999999999996,"15min":0.71899999999999997},"min":{"1min":0.51400000000000001,"5min":0.48199999999999998,"15min":0.434},"max":{"1min":0.70399999999999996,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.876},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70599999999999996,"15min":0.71299999999999997},"min":{"1min":0.48399999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.98099999999999998,"5min":1.143,"15min":1.149},"last":0.51400000000000001}]},{"osd":3,"last update":"Sun Jan 21 22:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70199999999999996,"15min":0.72799999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1419999999999999,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.754,"15min":0.72299999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.373},"max":{"1min":0.95099999999999996,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.73599999999999999}]},{"osd":4,"last update":"Sun Jan 21 22:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.33500000000000002,"15min":0.33600000000000002},"min":{"1min":0.29099999999999998,"5min":0.23599999999999999,"15min":0.22800000000000001},"max":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.34499999999999997,"15min":0.35099999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.25700000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.32500000000000001}]},{"osd":6,"last update":"Sun Jan 21 22:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.35199999999999998,"15min":0.35299999999999998},"min":{"1min":0.29999999999999999,"5min":0.26300000000000001,"15min":0.26000000000000001},"max":{"1min":0.42099999999999999,"5min":0.48899999999999999,"15min":0.51800000000000002},"last":0.376},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36599999999999999,"15min":0.36699999999999999},"min":{"1min":0.28100000000000003,"5min":0.27200000000000002,"15min":0.25600000000000001},"max":{"1min":0.442,"5min":0.48099999999999998,"15min":0.51100000000000001},"last":0.34300000000000003}]},{"osd":7,"last update":"Sun Jan 21 22:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.38300000000000001,"15min":0.38400000000000001},"min":{"1min":0.316,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38700000000000001,"15min":0.38500000000000001},"min":{"1min":0.311,"5min":0.27600000000000002,"15min":0.27100000000000002},"max":{"1min":0.51400000000000001,"5min":0.54100000000000004,"15min":0.57099999999999995},"last":0.42499999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542486,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65944,"kb_used_data":12560,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94305896,"statfs":{"total":96636764160,"available":96569237504,"internally_reserved":0,"allocated":12861440,"data_stored":12681197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.51100000000000001,"5min":0.46200000000000002,"15min":0.38800000000000001},"max":{"1min":0.97299999999999998,"5min":1.05,"15min":1.0880000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69399999999999995,"15min":0.71499999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.41099999999999998},"max":{"1min":0.998,"5min":1.008,"15min":2.9649999999999999},"last":0.60099999999999998}]},{"osd":1,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.70599999999999996,"15min":0.71899999999999997},"min":{"1min":0.45800000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.9410000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.70199999999999996,"15min":0.71399999999999997},"min":{"1min":0.44,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.615},"last":0.439}]},{"osd":2,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70499999999999996,"15min":0.71499999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.95799999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73499999999999999,"15min":0.71799999999999997},"min":{"1min":0.54700000000000004,"5min":0.505,"15min":0.40400000000000003},"max":{"1min":1.05,"5min":1.05,"15min":1.4710000000000001},"last":0.56100000000000005}]},{"osd":3,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69899999999999995,"15min":0.69799999999999995},"min":{"1min":0.55400000000000005,"5min":0.48499999999999999,"15min":0.43099999999999999},"max":{"1min":0.97499999999999998,"5min":1.0640000000000001,"15min":1.0720000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.72699999999999998,"15min":0.745},"min":{"1min":0.53800000000000003,"5min":0.52300000000000002,"15min":0.50900000000000001},"max":{"1min":0.92700000000000005,"5min":1.0600000000000001,"15min":1.1160000000000001},"last":0.41899999999999998}]},{"osd":5,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34499999999999997,"15min":0.34200000000000003},"min":{"1min":0.251,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.432,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.31},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.35099999999999998,"15min":0.34799999999999998},"min":{"1min":0.28000000000000003,"5min":0.252,"15min":0.249},"max":{"1min":0.42299999999999999,"5min":0.443,"15min":0.49399999999999999},"last":0.34399999999999997}]},{"osd":6,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34999999999999998,"15min":0.34699999999999998},"min":{"1min":0.28100000000000003,"5min":0.24299999999999999,"15min":0.23599999999999999},"max":{"1min":0.42099999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.378,"15min":0.374},"min":{"1min":0.29199999999999998,"5min":0.27700000000000002,"15min":0.26400000000000001},"max":{"1min":0.46800000000000003,"5min":0.51500000000000001,"15min":0.54000000000000004},"last":0.36799999999999999}]},{"osd":7,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.27600000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.52000000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39600000000000002,"15min":0.39000000000000001},"min":{"1min":0.317,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.47699999999999998,"5min":0.626,"15min":0.626},"last":0.39500000000000002}]}]},{"osd":3,"up_from":14,"seq":60129542487,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104396,"kb_used_data":37124,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94267444,"statfs":{"total":96636764160,"available":96529862656,"internally_reserved":0,"allocated":38014976,"data_stored":37846259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.093,"15min":1.0820000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.5150000000000001,"5min":3.9849999999999999,"15min":6.1879999999999997},"last":1.097},{"interface":"front","average":{"1min":1.302,"5min":1.2929999999999999,"15min":1.278},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":4.7430000000000003,"5min":8.6639999999999997,"15min":8.6639999999999997},"last":0.69499999999999995}]},{"osd":1,"last update":"Sun Jan 21 22:52:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.3109999999999999,"15min":1.26},"min":{"1min":0.77500000000000002,"5min":0.751,"15min":0.67000000000000004},"max":{"1min":3.8799999999999999,"5min":17.492999999999999,"15min":17.492999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.472,"15min":1.4139999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":5.7270000000000003,"5min":8.9629999999999992,"15min":8.9629999999999992},"last":1.246}]},{"osd":2,"last update":"Sun Jan 21 22:52:41 2024","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.4330000000000001,"15min":1.377},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":6.2119999999999997,"5min":8.5530000000000008,"15min":8.5530000000000008},"last":1.3},{"interface":"front","average":{"1min":1.484,"5min":1.581,"15min":1.5209999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":4.3049999999999997,"5min":17.664999999999999,"15min":17.664999999999999},"last":0.748}]},{"osd":4,"last update":"Sun Jan 21 22:52:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.1879999999999999,"15min":1.1479999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":4.226,"5min":4.5099999999999998,"15min":4.5099999999999998},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.2,"15min":1.1539999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":6.0110000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":1.151}]},{"osd":5,"last update":"Sun Jan 21 22:52:41 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.216,"15min":1.2010000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":9.2370000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.369,"5min":1.3799999999999999,"15min":1.3029999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":4.343,"5min":8.7420000000000009,"15min":8.7420000000000009},"last":1.46}]},{"osd":6,"last update":"Sun Jan 21 22:52:41 2024","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.373,"15min":1.3660000000000001},"min":{"1min":0.89800000000000002,"5min":0.874,"15min":0.85899999999999999},"max":{"1min":4.6230000000000002,"5min":4.6230000000000002,"15min":9.3719999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.423,"15min":1.3799999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":6.1109999999999998,"5min":6.1109999999999998,"15min":6.1109999999999998},"last":1.357}]},{"osd":7,"last update":"Sun Jan 21 22:52:41 2024","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.5069999999999999,"15min":1.498},"min":{"1min":1.1339999999999999,"5min":0.97199999999999998,"15min":0.86099999999999999},"max":{"1min":4.5359999999999996,"5min":4.5359999999999996,"15min":9.4610000000000003},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.5900000000000001,"15min":1.5169999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":5.242,"5min":8.8719999999999999,"15min":8.8719999999999999},"last":1.5109999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542485,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61776,"kb_used_data":12552,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94310064,"statfs":{"total":96636764160,"available":96573505536,"internally_reserved":0,"allocated":12853248,"data_stored":12680525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0509999999999999,"15min":1.04},"min":{"1min":0.79100000000000004,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5369999999999999,"5min":1.9410000000000001,"15min":2.0419999999999998},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.349,"5min":1.306,"15min":1.2769999999999999},"min":{"1min":0.95599999999999996,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.6299999999999999,"5min":2.2770000000000001,"15min":2.2770000000000001},"last":1.1559999999999999}]},{"osd":1,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.29,"15min":1.2629999999999999},"min":{"1min":0.88400000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.605,"5min":1.9399999999999999,"15min":2.234},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.3540000000000001,"15min":1.3759999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.8540000000000001,"5min":2.411,"15min":2.411},"last":1.3759999999999999}]},{"osd":3,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.27,"15min":1.296},"min":{"1min":0.96799999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.556,"5min":1.788,"15min":1.9690000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.429,"15min":1.4410000000000001},"min":{"1min":1.2230000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.681,"5min":2.0659999999999998,"15min":2.1320000000000001},"last":1.2030000000000001}]},{"osd":4,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.149,"15min":1.155},"min":{"1min":0.90100000000000002,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.345,"5min":1.776,"15min":2.0259999999999998},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0720000000000001,"15min":1.0820000000000001},"min":{"1min":0.69999999999999996,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4199999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":1.0489999999999999}]},{"osd":5,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.79900000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.415,"5min":1.623,"15min":1.669},"last":1.012},{"interface":"front","average":{"1min":1.244,"5min":1.2809999999999999,"15min":1.29},"min":{"1min":1.0860000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5720000000000001,"5min":1.8859999999999999,"15min":2.125},"last":1.1359999999999999}]},{"osd":6,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.244,"15min":1.25},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.6160000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.2130000000000001,"15min":1.254},"min":{"1min":0.96799999999999997,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.5880000000000001,"5min":1.8600000000000001,"15min":1.8919999999999999},"last":0.91600000000000004}]},{"osd":7,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.377,"15min":1.4019999999999999},"min":{"1min":1.034,"5min":0.748,"15min":0.748},"max":{"1min":1.7450000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":1.258},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3640000000000001,"15min":1.391},"min":{"1min":1.077,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.694,"5min":1.9139999999999999,"15min":2.004},"last":0.86599999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542486,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76576,"kb_used_data":23192,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94295264,"statfs":{"total":96636764160,"available":96558350336,"internally_reserved":0,"allocated":23748608,"data_stored":23575404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:48 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.05,"15min":1.044},"min":{"1min":0.70299999999999996,"5min":0.62,"15min":0.62},"max":{"1min":1.4450000000000001,"5min":2.3839999999999999,"15min":3.613},"last":0.75},{"interface":"front","average":{"1min":1.196,"5min":1.2290000000000001,"15min":1.2210000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.63800000000000001},"max":{"1min":1.4890000000000001,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":0.81599999999999995}]},{"osd":2,"last update":"Sun Jan 21 22:52:48 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.206,"15min":1.2350000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.6459999999999999,"5min":3.056,"15min":4.2309999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3480000000000001,"15min":1.3540000000000001},"min":{"1min":0.85899999999999999,"5min":0.82499999999999996,"15min":0.76700000000000002},"max":{"1min":1.6579999999999999,"5min":2.8159999999999998,"15min":3.2450000000000001},"last":0.88700000000000001}]},{"osd":3,"last update":"Sun Jan 21 22:52:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.421,"15min":1.4219999999999999},"min":{"1min":0.83999999999999997,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.605,"5min":4.0890000000000004,"15min":4.0890000000000004},"last":0.94699999999999995},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3400000000000001,"15min":1.355},"min":{"1min":1.0369999999999999,"5min":0.84599999999999997,"15min":0.80400000000000005},"max":{"1min":1.5649999999999999,"5min":3.8519999999999999,"15min":3.8519999999999999},"last":1.038}]},{"osd":4,"last update":"Sun Jan 21 22:52:48 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0629999999999999,"15min":1.038},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.4650000000000001,"5min":2.8639999999999999,"15min":2.8639999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.2030000000000001,"15min":1.1919999999999999},"min":{"1min":0.88200000000000001,"5min":0.78600000000000003,"15min":0.77300000000000002},"max":{"1min":1.526,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":1.0149999999999999}]},{"osd":5,"last update":"Sun Jan 21 22:52:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0980000000000001,"15min":1.1080000000000001},"min":{"1min":0.79400000000000004,"5min":0.71199999999999997,"15min":0.71099999999999997},"max":{"1min":1.6970000000000001,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.258,"15min":1.252},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.823,"5min":2.972,"15min":2.972},"last":0.90600000000000003}]},{"osd":6,"last update":"Sun Jan 21 22:52:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.282,"15min":1.294},"min":{"1min":0.92300000000000004,"5min":0.86799999999999999,"15min":0.79300000000000004},"max":{"1min":1.7969999999999999,"5min":3.121,"15min":3.121},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.401,"5min":1.417,"15min":1.4059999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.927,"5min":3.3410000000000002,"15min":3.3410000000000002},"last":0.98799999999999999}]},{"osd":7,"last update":"Sun Jan 21 22:52:48 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.415,"15min":1.4259999999999999},"min":{"1min":1.002,"5min":0.89100000000000001,"15min":0.86799999999999999},"max":{"1min":1.677,"5min":2.718,"15min":2.718},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.421,"15min":1.462},"min":{"1min":1.0289999999999999,"5min":1.002,"15min":0.93200000000000005},"max":{"1min":1.8560000000000001,"5min":3.218,"15min":3.218},"last":1.0780000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542487,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":93984,"kb_used_data":26712,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94277856,"statfs":{"total":96636764160,"available":96540524544,"internally_reserved":0,"allocated":27353088,"data_stored":27179326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0309999999999999,"15min":1.0640000000000001},"min":{"1min":0.70499999999999996,"5min":0.624,"15min":0.621},"max":{"1min":1.6080000000000001,"5min":1.6870000000000001,"15min":1.988},"last":0.751},{"interface":"front","average":{"1min":1.288,"5min":1.232,"15min":1.2889999999999999},"min":{"1min":0.90100000000000002,"5min":0.70199999999999996,"15min":0.67200000000000004},"max":{"1min":1.609,"5min":1.718,"15min":11.01},"last":1.379}]},{"osd":2,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.306,"15min":1.3300000000000001},"min":{"1min":1.2050000000000001,"5min":0.83799999999999997,"15min":0.81699999999999995},"max":{"1min":1.8580000000000001,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.3480000000000001,"15min":1.3879999999999999},"min":{"1min":0.96199999999999997,"5min":0.84499999999999997,"15min":0.84399999999999997},"max":{"1min":1.742,"5min":1.9690000000000001,"15min":11.308999999999999},"last":1.484}]},{"osd":3,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.268,"15min":1.3129999999999999},"min":{"1min":0.98999999999999999,"5min":0.85399999999999998,"15min":0.67000000000000004},"max":{"1min":1.573,"5min":1.671,"15min":9.2929999999999993},"last":1.462},{"interface":"front","average":{"1min":1.462,"5min":1.4390000000000001,"15min":1.4850000000000001},"min":{"1min":1.1679999999999999,"5min":1.01,"15min":0.91700000000000004},"max":{"1min":1.696,"5min":2.0390000000000001,"15min":11.4},"last":1.5700000000000001}]},{"osd":4,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0660000000000001,"15min":1.0860000000000001},"min":{"1min":0.82099999999999995,"5min":0.70999999999999996,"15min":0.66200000000000003},"max":{"1min":1.3200000000000001,"5min":1.593,"15min":2.0110000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.157,"15min":1.2050000000000001},"min":{"1min":0.94099999999999995,"5min":0.83899999999999997,"15min":0.77300000000000002},"max":{"1min":1.823,"5min":1.823,"15min":11.345000000000001},"last":1.139}]},{"osd":5,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.0289999999999999,"15min":1.0549999999999999},"min":{"1min":0.76200000000000001,"5min":0.71699999999999997,"15min":0.70299999999999996},"max":{"1min":1.498,"5min":1.7470000000000001,"15min":2.0030000000000001},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2190000000000001,"15min":1.266},"min":{"1min":0.84399999999999997,"5min":0.82799999999999996,"15min":0.66300000000000003},"max":{"1min":1.4099999999999999,"5min":1.6539999999999999,"15min":2.1509999999999998},"last":1.034}]},{"osd":6,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1819999999999999,"15min":1.2130000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.369,"5min":1.871,"15min":2.198},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.353,"5min":1.3540000000000001,"15min":1.397},"min":{"1min":1.0269999999999999,"5min":0.91000000000000003,"15min":0.872},"max":{"1min":1.74,"5min":1.8120000000000001,"15min":2.2509999999999999},"last":0.98599999999999999}]},{"osd":7,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.48,"5min":1.3680000000000001,"15min":1.4219999999999999},"min":{"1min":1.1839999999999999,"5min":0.96299999999999997,"15min":0.85799999999999998},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":11.374000000000001},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3460000000000001,"15min":1.369},"min":{"1min":0.97899999999999998,"5min":0.94999999999999996,"15min":0.875},"max":{"1min":1.855,"5min":1.9810000000000001,"15min":2.3319999999999999},"last":1.355}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":27090944,"data_stored":27082385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22892544,"data_stored":22888095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12591104,"data_stored":12583584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":37752832,"data_stored":37749318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12599296,"data_stored":12584256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":25788416,"data_stored":25776663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":13205504,"data_stored":13193783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-21T22:52:55.376 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.376 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.376 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.376 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.376 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.376 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.376 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.376 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.377 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.377 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.377 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.377 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.377 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.377 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.377 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.377 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.377 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.378 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.378 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.378 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.378 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.378 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.378 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.378 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.378 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.378 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.378 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.379 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.379 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.379 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.379 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-01-21T22:25:02.196506+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=2, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.379 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-01-21T22:25:02.196506+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=2, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.379 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.379 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-01-21T22:25:02.196506+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=2, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.379 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.380 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.380 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.380 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-01-21T22:25:02.196506+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=2, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.380 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.380 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.380 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-01-21T22:25:02.196506+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=2, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.380 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.381 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-01-21T22:25:02.196506+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=2, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.381 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.381 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.381 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-01-21T22:25:02.196506+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=2, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.381 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.381 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-01-21T22:25:02.196506+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=2, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.381 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.381 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-01-21T22:25:02.196506+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=2, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.381 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.382 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-01-21T22:25:02.196506+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=2, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.382 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.382 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-01-21T22:25:02.196506+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=2, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.382 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.382 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-01-21T22:25:02.196506+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=2, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.382 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.382 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-01-21T22:25:02.196506+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=2, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.382 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.382 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-01-21T22:25:02.196506+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=2, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.382 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-01-21T22:25:02.196506+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=2, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.383 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.383 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-01-21T22:25:02.196506+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=2, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.383 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.383 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-01-21T22:25:02.196506+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=2, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.383 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.383 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-01-21T22:25:02.196506+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=2, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.383 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.383 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-01-21T22:25:02.196506+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=2, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.383 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.384 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-01-21T22:25:02.196506+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=2, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.384 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.384 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.384 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.384 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-01-21T22:25:02.196506+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=2, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.384 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-01-21T22:25:03.444625+0000 time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=25, tm_sec=3, tm_wday=6, tm_yday=21, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=1, tm_mday=21, tm_hour=22, tm_min=52, tm_sec=45, tm_wday=6, tm_yday=21, tm_isdst=0) 2024-01-21T22:52:55.384 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-01-21T22:53:13.271 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:53:13.279 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:53:15.386 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-01-21T22:53:15.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.603+0000 7fd155e00640 1 Processor -- start 2024-01-21T22:53:15.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.603+0000 7fd155e00640 1 -- start start 2024-01-21T22:53:15.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.605+0000 7fd155e00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd150153a50 0x7fd1500c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:15.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.605+0000 7fd155e00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1500ca000 con 0x7fd15006b1c0 2024-01-21T22:53:15.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.605+0000 7fd155e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd15006ad30 con 0x7fd15006c0a0 2024-01-21T22:53:15.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.605+0000 7fd155e00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1500cb8d0 con 0x7fd150153a50 2024-01-21T22:53:15.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.606+0000 7fd154a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7fd150153a50 0x7fd1500c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:15.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.606+0000 7fd14fe00640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7fd15006c0a0 0x7fd1500cbb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:42378/0 (socket says 172.21.15.138:42378) 2024-01-21T22:53:15.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.606+0000 7fd14fe00640 1 -- 172.21.15.138:0/3128224549 learned_addr learned my addr 172.21.15.138:0/3128224549 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:53:15.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.606+0000 7fd155400640 1 --1- 172.21.15.138:0/3128224549 >> v1:172.21.15.138:6789/0 conn(0x7fd15006b1c0 0x7fd15006b630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:34976/0 (socket says 172.21.15.138:34976) 2024-01-21T22:53:15.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.609+0000 7fd14f400640 1 -- 172.21.15.138:0/3128224549 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418810790 0 0) 0x7fd15006ad30 con 0x7fd15006c0a0 2024-01-21T22:53:15.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.609+0000 7fd14f400640 1 -- 172.21.15.138:0/3128224549 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd130003520 con 0x7fd15006c0a0 2024-01-21T22:53:15.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.609+0000 7fd154a00640 1 -- 172.21.15.138:0/3128224549 >> v1:172.21.15.152:6789/0 conn(0x7fd15006c0a0 legacy=0x7fd1500cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:15.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.609+0000 7fd154a00640 1 -- 172.21.15.138:0/3128224549 >> v1:172.21.15.138:6789/0 conn(0x7fd15006b1c0 legacy=0x7fd15006b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:15.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.609+0000 7fd154a00640 1 -- 172.21.15.138:0/3128224549 --> v2:172.21.15.138:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd15015d120 con 0x7fd150153a50 2024-01-21T22:53:15.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.615+0000 7fd154a00640 1 --2- 172.21.15.138:0/3128224549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd150153a50 0x7fd1500c9ac0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd1400097a0 tx=0x7fd14002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f722ed484f8ea7b server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:15.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.619+0000 7fd14f400640 1 -- 172.21.15.138:0/3128224549 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd140031830 con 0x7fd150153a50 2024-01-21T22:53:15.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.619+0000 7fd14f400640 1 -- 172.21.15.138:0/3128224549 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd140031990 con 0x7fd150153a50 2024-01-21T22:53:15.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.619+0000 7fd14f400640 1 -- 172.21.15.138:0/3128224549 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd140031d20 con 0x7fd150153a50 2024-01-21T22:53:15.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.620+0000 7fd155e00640 1 -- 172.21.15.138:0/3128224549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd150153a50 msgr2=0x7fd1500c9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:15.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.620+0000 7fd155e00640 1 --2- 172.21.15.138:0/3128224549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd150153a50 0x7fd1500c9ac0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd1400097a0 tx=0x7fd14002b8d0 comp rx=0 tx=0).stop 2024-01-21T22:53:15.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.620+0000 7fd155e00640 1 -- 172.21.15.138:0/3128224549 shutdown_connections 2024-01-21T22:53:15.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.620+0000 7fd155e00640 1 --2- 172.21.15.138:0/3128224549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd150153a50 0x7fd1500c9ac0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:15.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.620+0000 7fd155e00640 1 -- 172.21.15.138:0/3128224549 >> 172.21.15.138:0/3128224549 conn(0x7fd15006d240 msgr2=0x7fd150066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:53:15.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.620+0000 7fd155e00640 1 -- 172.21.15.138:0/3128224549 shutdown_connections 2024-01-21T22:53:15.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.620+0000 7fd155e00640 1 -- 172.21.15.138:0/3128224549 wait complete. 2024-01-21T22:53:15.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.622+0000 7fd155e00640 1 Processor -- start 2024-01-21T22:53:15.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.622+0000 7fd155e00640 1 -- start start 2024-01-21T22:53:15.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.623+0000 7fd155e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd15006b1c0 0x7fd1500ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:15.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.623+0000 7fd155e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd15006c0a0 0x7fd1500c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:15.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.623+0000 7fd155e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd150153a50 0x7fd150146c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:15.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.623+0000 7fd155e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1500b4190 con 0x7fd150153a50 2024-01-21T22:53:15.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.623+0000 7fd155e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1500cf880 con 0x7fd15006b1c0 2024-01-21T22:53:15.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.623+0000 7fd155e00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1500c9dc0 con 0x7fd15006c0a0 2024-01-21T22:53:15.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.624+0000 7fd155400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd150153a50 0x7fd150146c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:15.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.624+0000 7fd154a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd15006b1c0 0x7fd1500ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:15.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.624+0000 7fd155400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd150153a50 0x7fd150146c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39372/0 (socket says 172.21.15.138:39372) 2024-01-21T22:53:15.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.624+0000 7fd155400640 1 -- 172.21.15.138:0/4104867062 learned_addr learned my addr 172.21.15.138:0/4104867062 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:53:15.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.624+0000 7fd14fe00640 1 --2- 172.21.15.138:0/4104867062 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd15006c0a0 0x7fd1500c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:15.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.628+0000 7fd155400640 1 -- 172.21.15.138:0/4104867062 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd15006c0a0 msgr2=0x7fd1500c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:15.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.628+0000 7fd155400640 1 --2- 172.21.15.138:0/4104867062 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd15006c0a0 0x7fd1500c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:15.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.628+0000 7fd155400640 1 -- 172.21.15.138:0/4104867062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd15006b1c0 msgr2=0x7fd1500ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:15.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.628+0000 7fd155400640 1 --2- 172.21.15.138:0/4104867062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd15006b1c0 0x7fd1500ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:15.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.628+0000 7fd155400640 1 -- 172.21.15.138:0/4104867062 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd140009400 con 0x7fd150153a50 2024-01-21T22:53:15.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.631+0000 7fd155400640 1 --2- 172.21.15.138:0/4104867062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd150153a50 0x7fd150146c80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd14800c730 tx=0x7fd14800cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:15.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.633+0000 7fd14cc00640 1 -- 172.21.15.138:0/4104867062 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd148004d60 con 0x7fd150153a50 2024-01-21T22:53:15.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.633+0000 7fd155e00640 1 -- 172.21.15.138:0/4104867062 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1500cfb40 con 0x7fd150153a50 2024-01-21T22:53:15.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.634+0000 7fd155e00640 1 -- 172.21.15.138:0/4104867062 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1500b3860 con 0x7fd150153a50 2024-01-21T22:53:15.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.635+0000 7fd155e00640 1 -- 172.21.15.138:0/4104867062 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd15004ea30 con 0x7fd150153a50 2024-01-21T22:53:15.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.636+0000 7fd14cc00640 1 -- 172.21.15.138:0/4104867062 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14800a640 con 0x7fd150153a50 2024-01-21T22:53:15.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.636+0000 7fd14cc00640 1 -- 172.21.15.138:0/4104867062 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14800a9b0 con 0x7fd150153a50 2024-01-21T22:53:15.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.941+0000 7fd14cc00640 1 -- 172.21.15.138:0/4104867062 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fd14800ab10 con 0x7fd150153a50 2024-01-21T22:53:15.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.943+0000 7fd14cc00640 1 --2- 172.21.15.138:0/4104867062 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd12806b360 0x7fd12806d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:15.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.944+0000 7fd154a00640 1 --2- 172.21.15.138:0/4104867062 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd12806b360 0x7fd12806d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:15.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.945+0000 7fd154a00640 1 --2- 172.21.15.138:0/4104867062 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd12806b360 0x7fd12806d820 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd140036000 tx=0x7fd14003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:15.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.974+0000 7fd14cc00640 1 -- 172.21.15.138:0/4104867062 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd148057fa0 con 0x7fd150153a50 2024-01-21T22:53:15.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:15.975+0000 7fd14cc00640 1 -- 172.21.15.138:0/4104867062 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fd1480542e0 con 0x7fd150153a50 2024-01-21T22:53:16.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.173+0000 7fd155e00640 1 -- 172.21.15.138:0/4104867062 --> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd15015d9c0 con 0x7fd12806b360 2024-01-21T22:53:16.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.182+0000 7fd14cc00640 1 -- 172.21.15.138:0/4104867062 <== mgr.4111 v2:172.21.15.152:6800/314625564 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290277 (secure 0 0 0) 0x7fd14003a420 con 0x7fd12806b360 2024-01-21T22:53:16.187 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:53:16.189 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-01-21T22:53:16.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.189+0000 7fd155e00640 1 -- 172.21.15.138:0/4104867062 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd12806b360 msgr2=0x7fd12806d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:16.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.189+0000 7fd155e00640 1 --2- 172.21.15.138:0/4104867062 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd12806b360 0x7fd12806d820 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd140036000 tx=0x7fd14003c040 comp rx=0 tx=0).stop 2024-01-21T22:53:16.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.189+0000 7fd155e00640 1 -- 172.21.15.138:0/4104867062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd150153a50 msgr2=0x7fd150146c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:16.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.189+0000 7fd155e00640 1 --2- 172.21.15.138:0/4104867062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd150153a50 0x7fd150146c80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd14800c730 tx=0x7fd14800cb30 comp rx=0 tx=0).stop 2024-01-21T22:53:16.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.189+0000 7fd155e00640 1 -- 172.21.15.138:0/4104867062 shutdown_connections 2024-01-21T22:53:16.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.189+0000 7fd155e00640 1 --2- 172.21.15.138:0/4104867062 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fd12806b360 0x7fd12806d820 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:16.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.189+0000 7fd155e00640 1 --2- 172.21.15.138:0/4104867062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd150153a50 0x7fd150146c80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:16.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.189+0000 7fd155e00640 1 --2- 172.21.15.138:0/4104867062 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fd15006c0a0 0x7fd1500c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:16.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.189+0000 7fd155e00640 1 --2- 172.21.15.138:0/4104867062 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fd15006b1c0 0x7fd1500ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:16.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.189+0000 7fd155e00640 1 -- 172.21.15.138:0/4104867062 >> 172.21.15.138:0/4104867062 conn(0x7fd15006d240 msgr2=0x7fd1500cbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:53:16.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.190+0000 7fd155e00640 1 -- 172.21.15.138:0/4104867062 shutdown_connections 2024-01-21T22:53:16.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.190+0000 7fd155e00640 1 -- 172.21.15.138:0/4104867062 wait complete. 2024-01-21T22:53:16.211 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":875,"stamp":"2024-01-21T22:53:15.545042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":82810866,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":0,"num_read":1695,"num_read_kb":212798,"num_write":26823,"num_write_kb":599405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28192,"ondisk_log_size":28192,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":610384,"kb_used_data":163856,"kb_used_omap":12,"kb_used_meta":446451,"kb_avail":754364336,"statfs":{"total":773094113280,"available":772469080064,"internally_reserved":0,"allocated":167788544,"data_stored":166397260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":457166385},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003022"},"pg_stats":[{"pgid":"3.3f","version":"25'165","reported_seq":209,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:02.981341+0000","last_change":"2024-01-21T22:53:02.981398+0000","last_active":"2024-01-21T22:53:02.981341+0000","last_peered":"2024-01-21T22:53:02.981341+0000","last_clean":"2024-01-21T22:53:02.981341+0000","last_became_active":"2024-01-21T22:25:04.888060+0000","last_became_peered":"2024-01-21T22:25:04.888060+0000","last_unstale":"2024-01-21T22:53:02.981341+0000","last_undegraded":"2024-01-21T22:53:02.981341+0000","last_fullsized":"2024-01-21T22:53:02.981341+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'165","last_scrub_stamp":"2024-01-21T22:53:02.981157+0000","last_deep_scrub":"25'165","last_deep_scrub_stamp":"2024-01-21T22:53:02.981157+0000","last_clean_scrub_stamp":"2024-01-21T22:53:02.981157+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:24:39.443463+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":115,"num_write_kb":6171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"25'170","reported_seq":219,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:02.451937+0000","last_change":"2024-01-21T22:53:02.451959+0000","last_active":"2024-01-21T22:53:02.451937+0000","last_peered":"2024-01-21T22:53:02.451937+0000","last_clean":"2024-01-21T22:53:02.451937+0000","last_became_active":"2024-01-21T22:25:04.843471+0000","last_became_peered":"2024-01-21T22:25:04.843471+0000","last_unstale":"2024-01-21T22:53:02.451937+0000","last_undegraded":"2024-01-21T22:53:02.451937+0000","last_fullsized":"2024-01-21T22:53:02.451937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'170","last_scrub_stamp":"2024-01-21T22:53:02.451862+0000","last_deep_scrub":"25'170","last_deep_scrub_stamp":"2024-01-21T22:53:02.451862+0000","last_clean_scrub_stamp":"2024-01-21T22:53:02.451862+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:20:54.428394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":129,"num_write_kb":4796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"25'202","reported_seq":250,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:04.466645+0000","last_change":"2024-01-21T22:53:04.466665+0000","last_active":"2024-01-21T22:53:04.466645+0000","last_peered":"2024-01-21T22:53:04.466645+0000","last_clean":"2024-01-21T22:53:04.466645+0000","last_became_active":"2024-01-21T22:25:04.857685+0000","last_became_peered":"2024-01-21T22:25:04.857685+0000","last_unstale":"2024-01-21T22:53:04.466645+0000","last_undegraded":"2024-01-21T22:53:04.466645+0000","last_fullsized":"2024-01-21T22:53:04.466645+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'202","last_scrub_stamp":"2024-01-21T22:53:04.466570+0000","last_deep_scrub":"25'202","last_deep_scrub_stamp":"2024-01-21T22:53:04.466570+0000","last_clean_scrub_stamp":"2024-01-21T22:53:04.466570+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:56:10.826047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":157,"num_write_kb":6223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"25'180","reported_seq":225,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:53.105348+0000","last_change":"2024-01-21T22:52:53.105348+0000","last_active":"2024-01-21T22:52:53.105348+0000","last_peered":"2024-01-21T22:52:53.105348+0000","last_clean":"2024-01-21T22:52:53.105348+0000","last_became_active":"2024-01-21T22:25:04.858278+0000","last_became_peered":"2024-01-21T22:25:04.858278+0000","last_unstale":"2024-01-21T22:52:53.105348+0000","last_undegraded":"2024-01-21T22:52:53.105348+0000","last_fullsized":"2024-01-21T22:52:53.105348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'180","last_scrub_stamp":"2024-01-21T22:52:53.105100+0000","last_deep_scrub":"25'180","last_deep_scrub_stamp":"2024-01-21T22:52:53.105100+0000","last_clean_scrub_stamp":"2024-01-21T22:52:53.105100+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:43:27.094985+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":135,"num_write_kb":19244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"25'181","reported_seq":227,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:54.917311+0000","last_change":"2024-01-21T22:52:54.917356+0000","last_active":"2024-01-21T22:52:54.917311+0000","last_peered":"2024-01-21T22:52:54.917311+0000","last_clean":"2024-01-21T22:52:54.917311+0000","last_became_active":"2024-01-21T22:25:04.878457+0000","last_became_peered":"2024-01-21T22:25:04.878457+0000","last_unstale":"2024-01-21T22:52:54.917311+0000","last_undegraded":"2024-01-21T22:52:54.917311+0000","last_fullsized":"2024-01-21T22:52:54.917311+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'181","last_scrub_stamp":"2024-01-21T22:52:54.917164+0000","last_deep_scrub":"25'181","last_deep_scrub_stamp":"2024-01-21T22:52:54.917164+0000","last_clean_scrub_stamp":"2024-01-21T22:52:54.917164+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:05:57.293663+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":119,"num_write_kb":7041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"25'199","reported_seq":236,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:54.117163+0000","last_change":"2024-01-21T22:52:54.117230+0000","last_active":"2024-01-21T22:52:54.117163+0000","last_peered":"2024-01-21T22:52:54.117163+0000","last_clean":"2024-01-21T22:52:54.117163+0000","last_became_active":"2024-01-21T22:25:04.858731+0000","last_became_peered":"2024-01-21T22:25:04.858731+0000","last_unstale":"2024-01-21T22:52:54.117163+0000","last_undegraded":"2024-01-21T22:52:54.117163+0000","last_fullsized":"2024-01-21T22:52:54.117163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'199","last_scrub_stamp":"2024-01-21T22:52:54.116980+0000","last_deep_scrub":"25'199","last_deep_scrub_stamp":"2024-01-21T22:52:54.116980+0000","last_clean_scrub_stamp":"2024-01-21T22:52:54.116980+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:39:29.796515+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":508,"num_write":148,"num_write_kb":2956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"25'212","reported_seq":265,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:56.947419+0000","last_change":"2024-01-21T22:52:56.947482+0000","last_active":"2024-01-21T22:52:56.947419+0000","last_peered":"2024-01-21T22:52:56.947419+0000","last_clean":"2024-01-21T22:52:56.947419+0000","last_became_active":"2024-01-21T22:25:04.881090+0000","last_became_peered":"2024-01-21T22:25:04.881090+0000","last_unstale":"2024-01-21T22:52:56.947419+0000","last_undegraded":"2024-01-21T22:52:56.947419+0000","last_fullsized":"2024-01-21T22:52:56.947419+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'212","last_scrub_stamp":"2024-01-21T22:52:56.947250+0000","last_deep_scrub":"25'212","last_deep_scrub_stamp":"2024-01-21T22:52:56.947250+0000","last_clean_scrub_stamp":"2024-01-21T22:52:56.947250+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:02:52.613626+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":4572,"num_write":149,"num_write_kb":5312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"25'174","reported_seq":222,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:06.548372+0000","last_change":"2024-01-21T22:53:06.548394+0000","last_active":"2024-01-21T22:53:06.548372+0000","last_peered":"2024-01-21T22:53:06.548372+0000","last_clean":"2024-01-21T22:53:06.548372+0000","last_became_active":"2024-01-21T22:25:04.878824+0000","last_became_peered":"2024-01-21T22:25:04.878824+0000","last_unstale":"2024-01-21T22:53:06.548372+0000","last_undegraded":"2024-01-21T22:53:06.548372+0000","last_fullsized":"2024-01-21T22:53:06.548372+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'174","last_scrub_stamp":"2024-01-21T22:53:06.548297+0000","last_deep_scrub":"25'174","last_deep_scrub_stamp":"2024-01-21T22:53:06.548297+0000","last_clean_scrub_stamp":"2024-01-21T22:53:06.548297+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:38:31.624777+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":137,"num_write_kb":11909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"25'162","reported_seq":204,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:53.928296+0000","last_change":"2024-01-21T22:52:53.928322+0000","last_active":"2024-01-21T22:52:53.928296+0000","last_peered":"2024-01-21T22:52:53.928296+0000","last_clean":"2024-01-21T22:52:53.928296+0000","last_became_active":"2024-01-21T22:25:04.884979+0000","last_became_peered":"2024-01-21T22:25:04.884979+0000","last_unstale":"2024-01-21T22:52:53.928296+0000","last_undegraded":"2024-01-21T22:52:53.928296+0000","last_fullsized":"2024-01-21T22:52:53.928296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'162","last_scrub_stamp":"2024-01-21T22:52:53.928218+0000","last_deep_scrub":"25'162","last_deep_scrub_stamp":"2024-01-21T22:52:53.928218+0000","last_clean_scrub_stamp":"2024-01-21T22:52:53.928218+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:57:50.255131+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":96,"num_write_kb":3924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"25'180","reported_seq":228,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:01.960408+0000","last_change":"2024-01-21T22:53:01.960466+0000","last_active":"2024-01-21T22:53:01.960408+0000","last_peered":"2024-01-21T22:53:01.960408+0000","last_clean":"2024-01-21T22:53:01.960408+0000","last_became_active":"2024-01-21T22:25:04.887631+0000","last_became_peered":"2024-01-21T22:25:04.887631+0000","last_unstale":"2024-01-21T22:53:01.960408+0000","last_undegraded":"2024-01-21T22:53:01.960408+0000","last_fullsized":"2024-01-21T22:53:01.960408+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'180","last_scrub_stamp":"2024-01-21T22:53:01.960225+0000","last_deep_scrub":"25'180","last_deep_scrub_stamp":"2024-01-21T22:53:01.960225+0000","last_clean_scrub_stamp":"2024-01-21T22:53:01.960225+0000","objects_scrubbed":0,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:13:29.980648+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":122,"num_write_kb":9871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"25'192","reported_seq":235,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:58.961876+0000","last_change":"2024-01-21T22:52:58.961984+0000","last_active":"2024-01-21T22:52:58.961876+0000","last_peered":"2024-01-21T22:52:58.961876+0000","last_clean":"2024-01-21T22:52:58.961876+0000","last_became_active":"2024-01-21T22:25:04.884697+0000","last_became_peered":"2024-01-21T22:25:04.884697+0000","last_unstale":"2024-01-21T22:52:58.961876+0000","last_undegraded":"2024-01-21T22:52:58.961876+0000","last_fullsized":"2024-01-21T22:52:58.961876+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'192","last_scrub_stamp":"2024-01-21T22:52:58.961642+0000","last_deep_scrub":"25'192","last_deep_scrub_stamp":"2024-01-21T22:52:58.961642+0000","last_clean_scrub_stamp":"2024-01-21T22:52:58.961642+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T06:10:18.987724+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":143,"num_write_kb":7172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"25'146","reported_seq":192,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:00.951544+0000","last_change":"2024-01-21T22:53:00.951634+0000","last_active":"2024-01-21T22:53:00.951544+0000","last_peered":"2024-01-21T22:53:00.951544+0000","last_clean":"2024-01-21T22:53:00.951544+0000","last_became_active":"2024-01-21T22:25:04.886387+0000","last_became_peered":"2024-01-21T22:25:04.886387+0000","last_unstale":"2024-01-21T22:53:00.951544+0000","last_undegraded":"2024-01-21T22:53:00.951544+0000","last_fullsized":"2024-01-21T22:53:00.951544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'146","last_scrub_stamp":"2024-01-21T22:53:00.951351+0000","last_deep_scrub":"25'146","last_deep_scrub_stamp":"2024-01-21T22:53:00.951351+0000","last_clean_scrub_stamp":"2024-01-21T22:53:00.951351+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:37:38.246832+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":92,"num_write_kb":3768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"25'169","reported_seq":211,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:01.731197+0000","last_change":"2024-01-21T22:53:01.731251+0000","last_active":"2024-01-21T22:53:01.731197+0000","last_peered":"2024-01-21T22:53:01.731197+0000","last_clean":"2024-01-21T22:53:01.731197+0000","last_became_active":"2024-01-21T22:25:04.887908+0000","last_became_peered":"2024-01-21T22:25:04.887908+0000","last_unstale":"2024-01-21T22:53:01.731197+0000","last_undegraded":"2024-01-21T22:53:01.731197+0000","last_fullsized":"2024-01-21T22:53:01.731197+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'169","last_scrub_stamp":"2024-01-21T22:53:01.731115+0000","last_deep_scrub":"25'169","last_deep_scrub_stamp":"2024-01-21T22:53:01.731115+0000","last_clean_scrub_stamp":"2024-01-21T22:53:01.731115+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:59:43.470420+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":122,"num_write_kb":8957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"25'174","reported_seq":223,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:58.623818+0000","last_change":"2024-01-21T22:52:58.623818+0000","last_active":"2024-01-21T22:52:58.623818+0000","last_peered":"2024-01-21T22:52:58.623818+0000","last_clean":"2024-01-21T22:52:58.623818+0000","last_became_active":"2024-01-21T22:25:04.842512+0000","last_became_peered":"2024-01-21T22:25:04.842512+0000","last_unstale":"2024-01-21T22:52:58.623818+0000","last_undegraded":"2024-01-21T22:52:58.623818+0000","last_fullsized":"2024-01-21T22:52:58.623818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'174","last_scrub_stamp":"2024-01-21T22:52:58.623687+0000","last_deep_scrub":"25'174","last_deep_scrub_stamp":"2024-01-21T22:52:58.623687+0000","last_clean_scrub_stamp":"2024-01-21T22:52:58.623687+0000","objects_scrubbed":0,"log_size":174,"log_dups_size":0,"ondisk_log_size":174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:17:26.972610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":120,"num_write_kb":6061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"25'170","reported_seq":236,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:05.507223+0000","last_change":"2024-01-21T22:53:05.507223+0000","last_active":"2024-01-21T22:53:05.507223+0000","last_peered":"2024-01-21T22:53:05.507223+0000","last_clean":"2024-01-21T22:53:05.507223+0000","last_became_active":"2024-01-21T22:25:04.876305+0000","last_became_peered":"2024-01-21T22:25:04.876305+0000","last_unstale":"2024-01-21T22:53:05.507223+0000","last_undegraded":"2024-01-21T22:53:05.507223+0000","last_fullsized":"2024-01-21T22:53:05.507223+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'170","last_scrub_stamp":"2024-01-21T22:53:05.507093+0000","last_deep_scrub":"25'170","last_deep_scrub_stamp":"2024-01-21T22:53:05.507093+0000","last_clean_scrub_stamp":"2024-01-21T22:53:05.507093+0000","objects_scrubbed":0,"log_size":170,"log_dups_size":0,"ondisk_log_size":170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T06:28:08.221114+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":7620,"num_write":119,"num_write_kb":10742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"25'209","reported_seq":265,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:04.500171+0000","last_change":"2024-01-21T22:53:04.500171+0000","last_active":"2024-01-21T22:53:04.500171+0000","last_peered":"2024-01-21T22:53:04.500171+0000","last_clean":"2024-01-21T22:53:04.500171+0000","last_became_active":"2024-01-21T22:25:04.877716+0000","last_became_peered":"2024-01-21T22:25:04.877716+0000","last_unstale":"2024-01-21T22:53:04.500171+0000","last_undegraded":"2024-01-21T22:53:04.500171+0000","last_fullsized":"2024-01-21T22:53:04.500171+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'209","last_scrub_stamp":"2024-01-21T22:53:04.499924+0000","last_deep_scrub":"25'209","last_deep_scrub_stamp":"2024-01-21T22:53:04.499924+0000","last_clean_scrub_stamp":"2024-01-21T22:53:04.499924+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:40:37.146723+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":5080,"num_write":159,"num_write_kb":7672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"25'183","reported_seq":230,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:08.404330+0000","last_change":"2024-01-21T22:53:08.404330+0000","last_active":"2024-01-21T22:53:08.404330+0000","last_peered":"2024-01-21T22:53:08.404330+0000","last_clean":"2024-01-21T22:53:08.404330+0000","last_became_active":"2024-01-21T22:25:04.890298+0000","last_became_peered":"2024-01-21T22:25:04.890298+0000","last_unstale":"2024-01-21T22:53:08.404330+0000","last_undegraded":"2024-01-21T22:53:08.404330+0000","last_fullsized":"2024-01-21T22:53:08.404330+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'183","last_scrub_stamp":"2024-01-21T22:53:08.404080+0000","last_deep_scrub":"25'183","last_deep_scrub_stamp":"2024-01-21T22:53:08.404080+0000","last_clean_scrub_stamp":"2024-01-21T22:53:08.404080+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:53:31.407789+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":136,"num_write_kb":7248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"25'159","reported_seq":204,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:57.583612+0000","last_change":"2024-01-21T22:52:57.583612+0000","last_active":"2024-01-21T22:52:57.583612+0000","last_peered":"2024-01-21T22:52:57.583612+0000","last_clean":"2024-01-21T22:52:57.583612+0000","last_became_active":"2024-01-21T22:25:04.712616+0000","last_became_peered":"2024-01-21T22:25:04.712616+0000","last_unstale":"2024-01-21T22:52:57.583612+0000","last_undegraded":"2024-01-21T22:52:57.583612+0000","last_fullsized":"2024-01-21T22:52:57.583612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'159","last_scrub_stamp":"2024-01-21T22:52:57.583511+0000","last_deep_scrub":"25'159","last_deep_scrub_stamp":"2024-01-21T22:52:57.583511+0000","last_clean_scrub_stamp":"2024-01-21T22:52:57.583511+0000","objects_scrubbed":0,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:25:54.478238+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":102,"num_write_kb":4431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"25'163","reported_seq":212,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:05.470335+0000","last_change":"2024-01-21T22:53:05.470395+0000","last_active":"2024-01-21T22:53:05.470335+0000","last_peered":"2024-01-21T22:53:05.470335+0000","last_clean":"2024-01-21T22:53:05.470335+0000","last_became_active":"2024-01-21T22:25:04.879011+0000","last_became_peered":"2024-01-21T22:25:04.879011+0000","last_unstale":"2024-01-21T22:53:05.470335+0000","last_undegraded":"2024-01-21T22:53:05.470335+0000","last_fullsized":"2024-01-21T22:53:05.470335+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'163","last_scrub_stamp":"2024-01-21T22:53:05.470155+0000","last_deep_scrub":"25'163","last_deep_scrub_stamp":"2024-01-21T22:53:05.470155+0000","last_clean_scrub_stamp":"2024-01-21T22:53:05.470155+0000","objects_scrubbed":0,"log_size":163,"log_dups_size":0,"ondisk_log_size":163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:47:25.321745+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":108,"num_write_kb":6073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"25'243","reported_seq":293,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:57.938782+0000","last_change":"2024-01-21T22:52:57.938839+0000","last_active":"2024-01-21T22:52:57.938782+0000","last_peered":"2024-01-21T22:52:57.938782+0000","last_clean":"2024-01-21T22:52:57.938782+0000","last_became_active":"2024-01-21T22:25:04.884364+0000","last_became_peered":"2024-01-21T22:25:04.884364+0000","last_unstale":"2024-01-21T22:52:57.938782+0000","last_undegraded":"2024-01-21T22:52:57.938782+0000","last_fullsized":"2024-01-21T22:52:57.938782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'243","last_scrub_stamp":"2024-01-21T22:52:57.938570+0000","last_deep_scrub":"25'243","last_deep_scrub_stamp":"2024-01-21T22:52:57.938570+0000","last_clean_scrub_stamp":"2024-01-21T22:52:57.938570+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:47:52.548870+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3556,"num_write":191,"num_write_kb":16584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"25'200","reported_seq":248,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:01.121844+0000","last_change":"2024-01-21T22:53:01.121868+0000","last_active":"2024-01-21T22:53:01.121844+0000","last_peered":"2024-01-21T22:53:01.121844+0000","last_clean":"2024-01-21T22:53:01.121844+0000","last_became_active":"2024-01-21T22:25:04.868196+0000","last_became_peered":"2024-01-21T22:25:04.868196+0000","last_unstale":"2024-01-21T22:53:01.121844+0000","last_undegraded":"2024-01-21T22:53:01.121844+0000","last_fullsized":"2024-01-21T22:53:01.121844+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'200","last_scrub_stamp":"2024-01-21T22:53:01.121767+0000","last_deep_scrub":"25'200","last_deep_scrub_stamp":"2024-01-21T22:53:01.121767+0000","last_clean_scrub_stamp":"2024-01-21T22:53:01.121767+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:49:22.034546+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":139,"num_write_kb":3732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"25'203","reported_seq":269,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:52.935450+0000","last_change":"2024-01-21T22:52:52.935491+0000","last_active":"2024-01-21T22:52:52.935450+0000","last_peered":"2024-01-21T22:52:52.935450+0000","last_clean":"2024-01-21T22:52:52.935450+0000","last_became_active":"2024-01-21T22:25:04.878868+0000","last_became_peered":"2024-01-21T22:25:04.878868+0000","last_unstale":"2024-01-21T22:52:52.935450+0000","last_undegraded":"2024-01-21T22:52:52.935450+0000","last_fullsized":"2024-01-21T22:52:52.935450+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'203","last_scrub_stamp":"2024-01-21T22:52:52.935365+0000","last_deep_scrub":"25'203","last_deep_scrub_stamp":"2024-01-21T22:52:52.935365+0000","last_clean_scrub_stamp":"2024-01-21T22:52:52.935365+0000","objects_scrubbed":0,"log_size":203,"log_dups_size":0,"ondisk_log_size":203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:46:06.748617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":7620,"num_write":153,"num_write_kb":6732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"25'173","reported_seq":218,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:58.436371+0000","last_change":"2024-01-21T22:52:58.436393+0000","last_active":"2024-01-21T22:52:58.436371+0000","last_peered":"2024-01-21T22:52:58.436371+0000","last_clean":"2024-01-21T22:52:58.436371+0000","last_became_active":"2024-01-21T22:25:04.714969+0000","last_became_peered":"2024-01-21T22:25:04.714969+0000","last_unstale":"2024-01-21T22:52:58.436371+0000","last_undegraded":"2024-01-21T22:52:58.436371+0000","last_fullsized":"2024-01-21T22:52:58.436371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'173","last_scrub_stamp":"2024-01-21T22:52:58.436300+0000","last_deep_scrub":"25'173","last_deep_scrub_stamp":"2024-01-21T22:52:58.436300+0000","last_clean_scrub_stamp":"2024-01-21T22:52:58.436300+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T10:41:01.867566+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":106,"num_write_kb":4123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"25'182","reported_seq":239,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:03.488817+0000","last_change":"2024-01-21T22:53:03.488925+0000","last_active":"2024-01-21T22:53:03.488817+0000","last_peered":"2024-01-21T22:53:03.488817+0000","last_clean":"2024-01-21T22:53:03.488817+0000","last_became_active":"2024-01-21T22:25:04.876963+0000","last_became_peered":"2024-01-21T22:25:04.876963+0000","last_unstale":"2024-01-21T22:53:03.488817+0000","last_undegraded":"2024-01-21T22:53:03.488817+0000","last_fullsized":"2024-01-21T22:53:03.488817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'182","last_scrub_stamp":"2024-01-21T22:53:03.488558+0000","last_deep_scrub":"25'182","last_deep_scrub_stamp":"2024-01-21T22:53:03.488558+0000","last_clean_scrub_stamp":"2024-01-21T22:53:03.488558+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:13:56.319507+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":5334,"num_write":137,"num_write_kb":8161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"25'184","reported_seq":243,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:01.404736+0000","last_change":"2024-01-21T22:53:01.404759+0000","last_active":"2024-01-21T22:53:01.404736+0000","last_peered":"2024-01-21T22:53:01.404736+0000","last_clean":"2024-01-21T22:53:01.404736+0000","last_became_active":"2024-01-21T22:25:04.716570+0000","last_became_peered":"2024-01-21T22:25:04.716570+0000","last_unstale":"2024-01-21T22:53:01.404736+0000","last_undegraded":"2024-01-21T22:53:01.404736+0000","last_fullsized":"2024-01-21T22:53:01.404736+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'184","last_scrub_stamp":"2024-01-21T22:53:01.404657+0000","last_deep_scrub":"25'184","last_deep_scrub_stamp":"2024-01-21T22:53:01.404657+0000","last_clean_scrub_stamp":"2024-01-21T22:53:01.404657+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:43:14.623961+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":5842,"num_write":145,"num_write_kb":5757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"25'176","reported_seq":226,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:56.156043+0000","last_change":"2024-01-21T22:52:56.156106+0000","last_active":"2024-01-21T22:52:56.156043+0000","last_peered":"2024-01-21T22:52:56.156043+0000","last_clean":"2024-01-21T22:52:56.156043+0000","last_became_active":"2024-01-21T22:25:04.885523+0000","last_became_peered":"2024-01-21T22:25:04.885523+0000","last_unstale":"2024-01-21T22:52:56.156043+0000","last_undegraded":"2024-01-21T22:52:56.156043+0000","last_fullsized":"2024-01-21T22:52:56.156043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'176","last_scrub_stamp":"2024-01-21T22:52:56.155850+0000","last_deep_scrub":"25'176","last_deep_scrub_stamp":"2024-01-21T22:52:56.155850+0000","last_clean_scrub_stamp":"2024-01-21T22:52:56.155850+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:29:37.670912+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3556,"num_write":106,"num_write_kb":5820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"25'213","reported_seq":258,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:56.546026+0000","last_change":"2024-01-21T22:52:56.546048+0000","last_active":"2024-01-21T22:52:56.546026+0000","last_peered":"2024-01-21T22:52:56.546026+0000","last_clean":"2024-01-21T22:52:56.546026+0000","last_became_active":"2024-01-21T22:25:04.842429+0000","last_became_peered":"2024-01-21T22:25:04.842429+0000","last_unstale":"2024-01-21T22:52:56.546026+0000","last_undegraded":"2024-01-21T22:52:56.546026+0000","last_fullsized":"2024-01-21T22:52:56.546026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'213","last_scrub_stamp":"2024-01-21T22:52:56.545952+0000","last_deep_scrub":"25'213","last_deep_scrub_stamp":"2024-01-21T22:52:56.545952+0000","last_clean_scrub_stamp":"2024-01-21T22:52:56.545952+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:01:46.762103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":167,"num_write_kb":10319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"25'183","reported_seq":231,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:02.770442+0000","last_change":"2024-01-21T22:53:02.770490+0000","last_active":"2024-01-21T22:53:02.770442+0000","last_peered":"2024-01-21T22:53:02.770442+0000","last_clean":"2024-01-21T22:53:02.770442+0000","last_became_active":"2024-01-21T22:25:04.888025+0000","last_became_peered":"2024-01-21T22:25:04.888025+0000","last_unstale":"2024-01-21T22:53:02.770442+0000","last_undegraded":"2024-01-21T22:53:02.770442+0000","last_fullsized":"2024-01-21T22:53:02.770442+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'183","last_scrub_stamp":"2024-01-21T22:53:02.770357+0000","last_deep_scrub":"25'183","last_deep_scrub_stamp":"2024-01-21T22:53:02.770357+0000","last_clean_scrub_stamp":"2024-01-21T22:53:02.770357+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:28:53.495300+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":136,"num_write_kb":6753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"25'199","reported_seq":248,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:56.428451+0000","last_change":"2024-01-21T22:52:56.428475+0000","last_active":"2024-01-21T22:52:56.428451+0000","last_peered":"2024-01-21T22:52:56.428451+0000","last_clean":"2024-01-21T22:52:56.428451+0000","last_became_active":"2024-01-21T22:25:04.711654+0000","last_became_peered":"2024-01-21T22:25:04.711654+0000","last_unstale":"2024-01-21T22:52:56.428451+0000","last_undegraded":"2024-01-21T22:52:56.428451+0000","last_fullsized":"2024-01-21T22:52:56.428451+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'199","last_scrub_stamp":"2024-01-21T22:52:56.428374+0000","last_deep_scrub":"25'199","last_deep_scrub_stamp":"2024-01-21T22:52:56.428374+0000","last_clean_scrub_stamp":"2024-01-21T22:52:56.428374+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:57:17.973368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":148,"num_write_kb":7328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"25'334","reported_seq":384,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:53.615625+0000","last_change":"2024-01-21T22:52:53.615693+0000","last_active":"2024-01-21T22:52:53.615625+0000","last_peered":"2024-01-21T22:52:53.615625+0000","last_clean":"2024-01-21T22:52:53.615625+0000","last_became_active":"2024-01-21T22:25:04.876012+0000","last_became_peered":"2024-01-21T22:25:04.876012+0000","last_unstale":"2024-01-21T22:52:53.615625+0000","last_undegraded":"2024-01-21T22:52:53.615625+0000","last_fullsized":"2024-01-21T22:52:53.615625+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'334","last_scrub_stamp":"2024-01-21T22:52:53.615359+0000","last_deep_scrub":"25'334","last_deep_scrub_stamp":"2024-01-21T22:52:53.615359+0000","last_clean_scrub_stamp":"2024-01-21T22:52:53.615359+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:55:59.974835+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":275,"num_write_kb":7140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"25'207","reported_seq":252,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:58.618336+0000","last_change":"2024-01-21T22:52:58.618360+0000","last_active":"2024-01-21T22:52:58.618336+0000","last_peered":"2024-01-21T22:52:58.618336+0000","last_clean":"2024-01-21T22:52:58.618336+0000","last_became_active":"2024-01-21T22:25:04.715034+0000","last_became_peered":"2024-01-21T22:25:04.715034+0000","last_unstale":"2024-01-21T22:52:58.618336+0000","last_undegraded":"2024-01-21T22:52:58.618336+0000","last_fullsized":"2024-01-21T22:52:58.618336+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'207","last_scrub_stamp":"2024-01-21T22:52:58.618265+0000","last_deep_scrub":"25'207","last_deep_scrub_stamp":"2024-01-21T22:52:58.618265+0000","last_clean_scrub_stamp":"2024-01-21T22:52:58.618265+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:46:12.431517+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":2032,"num_write":140,"num_write_kb":6049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"25'210","reported_seq":251,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:51.121042+0000","last_change":"2024-01-21T22:52:51.121096+0000","last_active":"2024-01-21T22:52:51.121042+0000","last_peered":"2024-01-21T22:52:51.121042+0000","last_clean":"2024-01-21T22:52:51.121042+0000","last_became_active":"2024-01-21T22:25:04.843827+0000","last_became_peered":"2024-01-21T22:25:04.843827+0000","last_unstale":"2024-01-21T22:52:51.121042+0000","last_undegraded":"2024-01-21T22:52:51.121042+0000","last_fullsized":"2024-01-21T22:52:51.121042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'210","last_scrub_stamp":"2024-01-21T22:52:51.120891+0000","last_deep_scrub":"25'210","last_deep_scrub_stamp":"2024-01-21T22:52:51.120891+0000","last_clean_scrub_stamp":"2024-01-21T22:52:51.120891+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:06:29.439012+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":1016,"num_write":149,"num_write_kb":12027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"25'158","reported_seq":204,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:50.887891+0000","last_change":"2024-01-21T22:52:50.887999+0000","last_active":"2024-01-21T22:52:50.887891+0000","last_peered":"2024-01-21T22:52:50.887891+0000","last_clean":"2024-01-21T22:52:50.887891+0000","last_became_active":"2024-01-21T22:25:04.712814+0000","last_became_peered":"2024-01-21T22:25:04.712814+0000","last_unstale":"2024-01-21T22:52:50.887891+0000","last_undegraded":"2024-01-21T22:52:50.887891+0000","last_fullsized":"2024-01-21T22:52:50.887891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'158","last_scrub_stamp":"2024-01-21T22:52:50.887622+0000","last_deep_scrub":"25'158","last_deep_scrub_stamp":"2024-01-21T22:52:50.887622+0000","last_clean_scrub_stamp":"2024-01-21T22:52:50.887622+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:13:27.478094+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":111,"num_write_kb":4009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"25'754","reported_seq":790,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:52.391023+0000","last_change":"2024-01-21T22:52:52.391050+0000","last_active":"2024-01-21T22:52:52.391023+0000","last_peered":"2024-01-21T22:52:52.391023+0000","last_clean":"2024-01-21T22:52:52.391023+0000","last_became_active":"2024-01-21T22:25:03.702776+0000","last_became_peered":"2024-01-21T22:25:03.702776+0000","last_unstale":"2024-01-21T22:52:52.391023+0000","last_undegraded":"2024-01-21T22:52:52.391023+0000","last_fullsized":"2024-01-21T22:52:52.391023+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'754","last_scrub_stamp":"2024-01-21T22:52:52.390950+0000","last_deep_scrub":"25'754","last_deep_scrub_stamp":"2024-01-21T22:52:52.390950+0000","last_clean_scrub_stamp":"2024-01-21T22:52:52.390950+0000","objects_scrubbed":1,"log_size":754,"log_dups_size":0,"ondisk_log_size":754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:17:20.739873+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":748,"num_write_kb":4569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"25'198","reported_seq":244,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:03.094365+0000","last_change":"2024-01-21T22:53:03.094388+0000","last_active":"2024-01-21T22:53:03.094365+0000","last_peered":"2024-01-21T22:53:03.094365+0000","last_clean":"2024-01-21T22:53:03.094365+0000","last_became_active":"2024-01-21T22:25:04.876920+0000","last_became_peered":"2024-01-21T22:25:04.876920+0000","last_unstale":"2024-01-21T22:53:03.094365+0000","last_undegraded":"2024-01-21T22:53:03.094365+0000","last_fullsized":"2024-01-21T22:53:03.094365+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'198","last_scrub_stamp":"2024-01-21T22:53:03.094260+0000","last_deep_scrub":"25'198","last_deep_scrub_stamp":"2024-01-21T22:53:03.094260+0000","last_clean_scrub_stamp":"2024-01-21T22:53:03.094260+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:14:03.420251+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":2794,"num_write":152,"num_write_kb":6158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"25'777","reported_seq":813,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:02.540743+0000","last_change":"2024-01-21T22:53:02.540817+0000","last_active":"2024-01-21T22:53:02.540743+0000","last_peered":"2024-01-21T22:53:02.540743+0000","last_clean":"2024-01-21T22:53:02.540743+0000","last_became_active":"2024-01-21T22:25:03.719909+0000","last_became_peered":"2024-01-21T22:25:03.719909+0000","last_unstale":"2024-01-21T22:53:02.540743+0000","last_undegraded":"2024-01-21T22:53:02.540743+0000","last_fullsized":"2024-01-21T22:53:02.540743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'777","last_scrub_stamp":"2024-01-21T22:53:02.540526+0000","last_deep_scrub":"25'777","last_deep_scrub_stamp":"2024-01-21T22:53:02.540526+0000","last_clean_scrub_stamp":"2024-01-21T22:53:02.540526+0000","objects_scrubbed":1,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T10:10:04.898159+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":769,"num_write_kb":4580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"25'3","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:55.507540+0000","last_change":"2024-01-21T22:52:55.507579+0000","last_active":"2024-01-21T22:52:55.507540+0000","last_peered":"2024-01-21T22:52:55.507540+0000","last_clean":"2024-01-21T22:52:55.507540+0000","last_became_active":"2024-01-21T22:25:03.722758+0000","last_became_peered":"2024-01-21T22:25:03.722758+0000","last_unstale":"2024-01-21T22:52:55.507540+0000","last_undegraded":"2024-01-21T22:52:55.507540+0000","last_fullsized":"2024-01-21T22:52:55.507540+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'3","last_scrub_stamp":"2024-01-21T22:52:55.507422+0000","last_deep_scrub":"25'3","last_deep_scrub_stamp":"2024-01-21T22:52:55.507422+0000","last_clean_scrub_stamp":"2024-01-21T22:52:55.507422+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:57:51.834335+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"25'224","reported_seq":272,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:03.429930+0000","last_change":"2024-01-21T22:53:03.429954+0000","last_active":"2024-01-21T22:53:03.429930+0000","last_peered":"2024-01-21T22:53:03.429930+0000","last_clean":"2024-01-21T22:53:03.429930+0000","last_became_active":"2024-01-21T22:25:04.843714+0000","last_became_peered":"2024-01-21T22:25:04.843714+0000","last_unstale":"2024-01-21T22:53:03.429930+0000","last_undegraded":"2024-01-21T22:53:03.429930+0000","last_fullsized":"2024-01-21T22:53:03.429930+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'224","last_scrub_stamp":"2024-01-21T22:53:03.429843+0000","last_deep_scrub":"25'224","last_deep_scrub_stamp":"2024-01-21T22:53:03.429843+0000","last_clean_scrub_stamp":"2024-01-21T22:53:03.429843+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:23:29.691220+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":174,"num_write_kb":5546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"25'1197","reported_seq":1233,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:56.629296+0000","last_change":"2024-01-21T22:52:56.629358+0000","last_active":"2024-01-21T22:52:56.629296+0000","last_peered":"2024-01-21T22:52:56.629296+0000","last_clean":"2024-01-21T22:52:56.629296+0000","last_became_active":"2024-01-21T22:25:03.702140+0000","last_became_peered":"2024-01-21T22:25:03.702140+0000","last_unstale":"2024-01-21T22:52:56.629296+0000","last_undegraded":"2024-01-21T22:52:56.629296+0000","last_fullsized":"2024-01-21T22:52:56.629296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1197","last_scrub_stamp":"2024-01-21T22:52:56.629135+0000","last_deep_scrub":"25'1197","last_deep_scrub_stamp":"2024-01-21T22:52:56.629135+0000","last_clean_scrub_stamp":"2024-01-21T22:52:56.629135+0000","objects_scrubbed":2,"log_size":1197,"log_dups_size":0,"ondisk_log_size":1197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:44:01.951581+0000","scrub_duration":62,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1190,"num_write_kb":8787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"25'199","reported_seq":246,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:00.124092+0000","last_change":"2024-01-21T22:53:00.124137+0000","last_active":"2024-01-21T22:53:00.124092+0000","last_peered":"2024-01-21T22:53:00.124092+0000","last_clean":"2024-01-21T22:53:00.124092+0000","last_became_active":"2024-01-21T22:25:04.866945+0000","last_became_peered":"2024-01-21T22:25:04.866945+0000","last_unstale":"2024-01-21T22:53:00.124092+0000","last_undegraded":"2024-01-21T22:53:00.124092+0000","last_fullsized":"2024-01-21T22:53:00.124092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'199","last_scrub_stamp":"2024-01-21T22:53:00.124006+0000","last_deep_scrub":"25'199","last_deep_scrub_stamp":"2024-01-21T22:53:00.124006+0000","last_clean_scrub_stamp":"2024-01-21T22:53:00.124006+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:50:15.982355+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":136,"num_write_kb":8996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"25'3","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:49.897688+0000","last_change":"2024-01-21T22:52:49.897754+0000","last_active":"2024-01-21T22:52:49.897688+0000","last_peered":"2024-01-21T22:52:49.897688+0000","last_clean":"2024-01-21T22:52:49.897688+0000","last_became_active":"2024-01-21T22:25:03.714039+0000","last_became_peered":"2024-01-21T22:25:03.714039+0000","last_unstale":"2024-01-21T22:52:49.897688+0000","last_undegraded":"2024-01-21T22:52:49.897688+0000","last_fullsized":"2024-01-21T22:52:49.897688+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'3","last_scrub_stamp":"2024-01-21T22:52:49.897484+0000","last_deep_scrub":"25'3","last_deep_scrub_stamp":"2024-01-21T22:52:49.897484+0000","last_clean_scrub_stamp":"2024-01-21T22:52:49.897484+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T10:44:49.265288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"25'210","reported_seq":257,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:57.163330+0000","last_change":"2024-01-21T22:52:57.163388+0000","last_active":"2024-01-21T22:52:57.163330+0000","last_peered":"2024-01-21T22:52:57.163330+0000","last_clean":"2024-01-21T22:52:57.163330+0000","last_became_active":"2024-01-21T22:25:04.885963+0000","last_became_peered":"2024-01-21T22:25:04.885963+0000","last_unstale":"2024-01-21T22:52:57.163330+0000","last_undegraded":"2024-01-21T22:52:57.163330+0000","last_fullsized":"2024-01-21T22:52:57.163330+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'210","last_scrub_stamp":"2024-01-21T22:52:57.163154+0000","last_deep_scrub":"25'210","last_deep_scrub_stamp":"2024-01-21T22:52:57.163154+0000","last_clean_scrub_stamp":"2024-01-21T22:52:57.163154+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-22T23:00:18.847582+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":149,"num_write_kb":6741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"25'5","reported_seq":41,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:48.887640+0000","last_change":"2024-01-21T22:52:48.887744+0000","last_active":"2024-01-21T22:52:48.887640+0000","last_peered":"2024-01-21T22:52:48.887640+0000","last_clean":"2024-01-21T22:52:48.887640+0000","last_became_active":"2024-01-21T22:25:03.717937+0000","last_became_peered":"2024-01-21T22:25:03.717937+0000","last_unstale":"2024-01-21T22:52:48.887640+0000","last_undegraded":"2024-01-21T22:52:48.887640+0000","last_fullsized":"2024-01-21T22:52:48.887640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'5","last_scrub_stamp":"2024-01-21T22:52:48.887398+0000","last_deep_scrub":"25'5","last_deep_scrub_stamp":"2024-01-21T22:52:48.887398+0000","last_clean_scrub_stamp":"2024-01-21T22:52:48.887398+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:19:29.088565+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"25'195","reported_seq":240,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:07.516001+0000","last_change":"2024-01-21T22:53:07.516024+0000","last_active":"2024-01-21T22:53:07.516001+0000","last_peered":"2024-01-21T22:53:07.516001+0000","last_clean":"2024-01-21T22:53:07.516001+0000","last_became_active":"2024-01-21T22:25:04.882478+0000","last_became_peered":"2024-01-21T22:25:04.882478+0000","last_unstale":"2024-01-21T22:53:07.516001+0000","last_undegraded":"2024-01-21T22:53:07.516001+0000","last_fullsized":"2024-01-21T22:53:07.516001+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'195","last_scrub_stamp":"2024-01-21T22:53:07.515914+0000","last_deep_scrub":"25'195","last_deep_scrub_stamp":"2024-01-21T22:53:07.515914+0000","last_clean_scrub_stamp":"2024-01-21T22:53:07.515914+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:44:12.838360+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2540,"num_write":148,"num_write_kb":5415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"25'469","reported_seq":505,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:54.552226+0000","last_change":"2024-01-21T22:52:54.552270+0000","last_active":"2024-01-21T22:52:54.552226+0000","last_peered":"2024-01-21T22:52:54.552226+0000","last_clean":"2024-01-21T22:52:54.552226+0000","last_became_active":"2024-01-21T22:25:03.703426+0000","last_became_peered":"2024-01-21T22:25:03.703426+0000","last_unstale":"2024-01-21T22:52:54.552226+0000","last_undegraded":"2024-01-21T22:52:54.552226+0000","last_fullsized":"2024-01-21T22:52:54.552226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'469","last_scrub_stamp":"2024-01-21T22:52:54.552135+0000","last_deep_scrub":"25'469","last_deep_scrub_stamp":"2024-01-21T22:52:54.552135+0000","last_clean_scrub_stamp":"2024-01-21T22:52:54.552135+0000","objects_scrubbed":2,"log_size":469,"log_dups_size":0,"ondisk_log_size":469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:14:19.685541+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":467,"num_write_kb":464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"25'215","reported_seq":273,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:07.438574+0000","last_change":"2024-01-21T22:53:07.438574+0000","last_active":"2024-01-21T22:53:07.438574+0000","last_peered":"2024-01-21T22:53:07.438574+0000","last_clean":"2024-01-21T22:53:07.438574+0000","last_became_active":"2024-01-21T22:25:04.886970+0000","last_became_peered":"2024-01-21T22:25:04.886970+0000","last_unstale":"2024-01-21T22:53:07.438574+0000","last_undegraded":"2024-01-21T22:53:07.438574+0000","last_fullsized":"2024-01-21T22:53:07.438574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'215","last_scrub_stamp":"2024-01-21T22:53:07.438317+0000","last_deep_scrub":"25'215","last_deep_scrub_stamp":"2024-01-21T22:53:07.438317+0000","last_clean_scrub_stamp":"2024-01-21T22:53:07.438317+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:52:01.597871+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":5588,"num_write":173,"num_write_kb":7144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"25'882","reported_seq":919,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:50.132827+0000","last_change":"2024-01-21T22:52:50.132827+0000","last_active":"2024-01-21T22:52:50.132827+0000","last_peered":"2024-01-21T22:52:50.132827+0000","last_clean":"2024-01-21T22:52:50.132827+0000","last_became_active":"2024-01-21T22:25:03.713717+0000","last_became_peered":"2024-01-21T22:25:03.713717+0000","last_unstale":"2024-01-21T22:52:50.132827+0000","last_undegraded":"2024-01-21T22:52:50.132827+0000","last_fullsized":"2024-01-21T22:52:50.132827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'882","last_scrub_stamp":"2024-01-21T22:52:50.132664+0000","last_deep_scrub":"25'882","last_deep_scrub_stamp":"2024-01-21T22:52:50.132664+0000","last_clean_scrub_stamp":"2024-01-21T22:52:50.132664+0000","objects_scrubbed":4,"log_size":882,"log_dups_size":0,"ondisk_log_size":882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:56:28.261981+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":877,"num_write_kb":4705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"25'169","reported_seq":222,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:55.943009+0000","last_change":"2024-01-21T22:52:55.943071+0000","last_active":"2024-01-21T22:52:55.943009+0000","last_peered":"2024-01-21T22:52:55.943009+0000","last_clean":"2024-01-21T22:52:55.943009+0000","last_became_active":"2024-01-21T22:25:04.880472+0000","last_became_peered":"2024-01-21T22:25:04.880472+0000","last_unstale":"2024-01-21T22:52:55.943009+0000","last_undegraded":"2024-01-21T22:52:55.943009+0000","last_fullsized":"2024-01-21T22:52:55.943009+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'169","last_scrub_stamp":"2024-01-21T22:52:55.942817+0000","last_deep_scrub":"25'169","last_deep_scrub_stamp":"2024-01-21T22:52:55.942817+0000","last_clean_scrub_stamp":"2024-01-21T22:52:55.942817+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T09:20:47.919349+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":4572,"num_write":122,"num_write_kb":6796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"25'1294","reported_seq":1331,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:56.605874+0000","last_change":"2024-01-21T22:52:56.605874+0000","last_active":"2024-01-21T22:52:56.605874+0000","last_peered":"2024-01-21T22:52:56.605874+0000","last_clean":"2024-01-21T22:52:56.605874+0000","last_became_active":"2024-01-21T22:25:03.721502+0000","last_became_peered":"2024-01-21T22:25:03.721502+0000","last_unstale":"2024-01-21T22:52:56.605874+0000","last_undegraded":"2024-01-21T22:52:56.605874+0000","last_fullsized":"2024-01-21T22:52:56.605874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1294","last_scrub_stamp":"2024-01-21T22:52:56.605777+0000","last_deep_scrub":"25'1294","last_deep_scrub_stamp":"2024-01-21T22:52:56.605777+0000","last_clean_scrub_stamp":"2024-01-21T22:52:56.605777+0000","objects_scrubbed":4,"log_size":1294,"log_dups_size":0,"ondisk_log_size":1294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:55:12.954521+0000","scrub_duration":62,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388642,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1286,"num_write_kb":8849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"25'164","reported_seq":205,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:00.698169+0000","last_change":"2024-01-21T22:53:00.698169+0000","last_active":"2024-01-21T22:53:00.698169+0000","last_peered":"2024-01-21T22:53:00.698169+0000","last_clean":"2024-01-21T22:53:00.698169+0000","last_became_active":"2024-01-21T22:25:04.877257+0000","last_became_peered":"2024-01-21T22:25:04.877257+0000","last_unstale":"2024-01-21T22:53:00.698169+0000","last_undegraded":"2024-01-21T22:53:00.698169+0000","last_fullsized":"2024-01-21T22:53:00.698169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'164","last_scrub_stamp":"2024-01-21T22:53:00.698066+0000","last_deep_scrub":"25'164","last_deep_scrub_stamp":"2024-01-21T22:53:00.698066+0000","last_clean_scrub_stamp":"2024-01-21T22:53:00.698066+0000","objects_scrubbed":0,"log_size":164,"log_dups_size":0,"ondisk_log_size":164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T10:25:07.803176+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":1270,"num_write":117,"num_write_kb":4465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"25'5","reported_seq":42,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:58.575504+0000","last_change":"2024-01-21T22:52:58.575504+0000","last_active":"2024-01-21T22:52:58.575504+0000","last_peered":"2024-01-21T22:52:58.575504+0000","last_clean":"2024-01-21T22:52:58.575504+0000","last_became_active":"2024-01-21T22:25:03.702798+0000","last_became_peered":"2024-01-21T22:25:03.702798+0000","last_unstale":"2024-01-21T22:52:58.575504+0000","last_undegraded":"2024-01-21T22:52:58.575504+0000","last_fullsized":"2024-01-21T22:52:58.575504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'5","last_scrub_stamp":"2024-01-21T22:52:58.575265+0000","last_deep_scrub":"25'5","last_deep_scrub_stamp":"2024-01-21T22:52:58.575265+0000","last_clean_scrub_stamp":"2024-01-21T22:52:58.575265+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:33:30.334848+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"25'231","reported_seq":276,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:09.415783+0000","last_change":"2024-01-21T22:53:09.415890+0000","last_active":"2024-01-21T22:53:09.415783+0000","last_peered":"2024-01-21T22:53:09.415783+0000","last_clean":"2024-01-21T22:53:09.415783+0000","last_became_active":"2024-01-21T22:25:04.890271+0000","last_became_peered":"2024-01-21T22:25:04.890271+0000","last_unstale":"2024-01-21T22:53:09.415783+0000","last_undegraded":"2024-01-21T22:53:09.415783+0000","last_fullsized":"2024-01-21T22:53:09.415783+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'231","last_scrub_stamp":"2024-01-21T22:53:09.415486+0000","last_deep_scrub":"25'231","last_deep_scrub_stamp":"2024-01-21T22:53:09.415486+0000","last_clean_scrub_stamp":"2024-01-21T22:53:09.415486+0000","objects_scrubbed":0,"log_size":231,"log_dups_size":0,"ondisk_log_size":231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:58:15.215232+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":174,"num_write_kb":31551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"25'4","reported_seq":41,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:47.913624+0000","last_change":"2024-01-21T22:52:47.913624+0000","last_active":"2024-01-21T22:52:47.913624+0000","last_peered":"2024-01-21T22:52:47.913624+0000","last_clean":"2024-01-21T22:52:47.913624+0000","last_became_active":"2024-01-21T22:25:03.716846+0000","last_became_peered":"2024-01-21T22:25:03.716846+0000","last_unstale":"2024-01-21T22:52:47.913624+0000","last_undegraded":"2024-01-21T22:52:47.913624+0000","last_fullsized":"2024-01-21T22:52:47.913624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'4","last_scrub_stamp":"2024-01-21T22:52:47.913345+0000","last_deep_scrub":"25'4","last_deep_scrub_stamp":"2024-01-21T22:52:47.913345+0000","last_clean_scrub_stamp":"2024-01-21T22:52:47.913345+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:28:57.508230+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"25'169","reported_seq":212,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:51.897302+0000","last_change":"2024-01-21T22:52:51.897302+0000","last_active":"2024-01-21T22:52:51.897302+0000","last_peered":"2024-01-21T22:52:51.897302+0000","last_clean":"2024-01-21T22:52:51.897302+0000","last_became_active":"2024-01-21T22:25:04.713566+0000","last_became_peered":"2024-01-21T22:25:04.713566+0000","last_unstale":"2024-01-21T22:52:51.897302+0000","last_undegraded":"2024-01-21T22:52:51.897302+0000","last_fullsized":"2024-01-21T22:52:51.897302+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'169","last_scrub_stamp":"2024-01-21T22:52:51.897088+0000","last_deep_scrub":"25'169","last_deep_scrub_stamp":"2024-01-21T22:52:51.897088+0000","last_clean_scrub_stamp":"2024-01-21T22:52:51.897088+0000","objects_scrubbed":0,"log_size":169,"log_dups_size":0,"ondisk_log_size":169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T10:27:33.467202+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":119,"num_write_kb":3558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"25'14","reported_seq":51,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:53.234100+0000","last_change":"2024-01-21T22:52:53.234100+0000","last_active":"2024-01-21T22:52:53.234100+0000","last_peered":"2024-01-21T22:52:53.234100+0000","last_clean":"2024-01-21T22:52:53.234100+0000","last_became_active":"2024-01-21T22:25:03.702742+0000","last_became_peered":"2024-01-21T22:25:03.702742+0000","last_unstale":"2024-01-21T22:52:53.234100+0000","last_undegraded":"2024-01-21T22:52:53.234100+0000","last_fullsized":"2024-01-21T22:52:53.234100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'14","last_scrub_stamp":"2024-01-21T22:52:53.233961+0000","last_deep_scrub":"25'14","last_deep_scrub_stamp":"2024-01-21T22:52:53.233961+0000","last_clean_scrub_stamp":"2024-01-21T22:52:53.233961+0000","objects_scrubbed":4,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:01:43.450112+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"25'196","reported_seq":239,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:51.598050+0000","last_change":"2024-01-21T22:52:51.598050+0000","last_active":"2024-01-21T22:52:51.598050+0000","last_peered":"2024-01-21T22:52:51.598050+0000","last_clean":"2024-01-21T22:52:51.598050+0000","last_became_active":"2024-01-21T22:25:04.876169+0000","last_became_peered":"2024-01-21T22:25:04.876169+0000","last_unstale":"2024-01-21T22:52:51.598050+0000","last_undegraded":"2024-01-21T22:52:51.598050+0000","last_fullsized":"2024-01-21T22:52:51.598050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'196","last_scrub_stamp":"2024-01-21T22:52:51.597942+0000","last_deep_scrub":"25'196","last_deep_scrub_stamp":"2024-01-21T22:52:51.597942+0000","last_clean_scrub_stamp":"2024-01-21T22:52:51.597942+0000","objects_scrubbed":0,"log_size":196,"log_dups_size":0,"ondisk_log_size":196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:04:02.551018+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":142,"num_write_kb":8431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"25'8","reported_seq":45,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:52.936783+0000","last_change":"2024-01-21T22:52:52.936813+0000","last_active":"2024-01-21T22:52:52.936783+0000","last_peered":"2024-01-21T22:52:52.936783+0000","last_clean":"2024-01-21T22:52:52.936783+0000","last_became_active":"2024-01-21T22:25:03.720009+0000","last_became_peered":"2024-01-21T22:25:03.720009+0000","last_unstale":"2024-01-21T22:52:52.936783+0000","last_undegraded":"2024-01-21T22:52:52.936783+0000","last_fullsized":"2024-01-21T22:52:52.936783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'8","last_scrub_stamp":"2024-01-21T22:52:52.936704+0000","last_deep_scrub":"25'8","last_deep_scrub_stamp":"2024-01-21T22:52:52.936704+0000","last_clean_scrub_stamp":"2024-01-21T22:52:52.936704+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:13:53.262696+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"25'153","reported_seq":210,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:55.145252+0000","last_change":"2024-01-21T22:52:55.145310+0000","last_active":"2024-01-21T22:52:55.145252+0000","last_peered":"2024-01-21T22:52:55.145252+0000","last_clean":"2024-01-21T22:52:55.145252+0000","last_became_active":"2024-01-21T22:25:04.882767+0000","last_became_peered":"2024-01-21T22:25:04.882767+0000","last_unstale":"2024-01-21T22:52:55.145252+0000","last_undegraded":"2024-01-21T22:52:55.145252+0000","last_fullsized":"2024-01-21T22:52:55.145252+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'153","last_scrub_stamp":"2024-01-21T22:52:55.145061+0000","last_deep_scrub":"25'153","last_deep_scrub_stamp":"2024-01-21T22:52:55.145061+0000","last_clean_scrub_stamp":"2024-01-21T22:52:55.145061+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T06:47:45.644151+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":5334,"num_write":115,"num_write_kb":10137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"25'302","reported_seq":339,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:54.285150+0000","last_change":"2024-01-21T22:52:54.285204+0000","last_active":"2024-01-21T22:52:54.285150+0000","last_peered":"2024-01-21T22:52:54.285150+0000","last_clean":"2024-01-21T22:52:54.285150+0000","last_became_active":"2024-01-21T22:25:03.703256+0000","last_became_peered":"2024-01-21T22:25:03.703256+0000","last_unstale":"2024-01-21T22:52:54.285150+0000","last_undegraded":"2024-01-21T22:52:54.285150+0000","last_fullsized":"2024-01-21T22:52:54.285150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'302","last_scrub_stamp":"2024-01-21T22:52:54.285072+0000","last_deep_scrub":"25'302","last_deep_scrub_stamp":"2024-01-21T22:52:54.285072+0000","last_clean_scrub_stamp":"2024-01-21T22:52:54.285072+0000","objects_scrubbed":2,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T10:34:57.974916+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":296,"num_write_kb":4241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"25'166","reported_seq":209,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:00.611723+0000","last_change":"2024-01-21T22:53:00.611761+0000","last_active":"2024-01-21T22:53:00.611723+0000","last_peered":"2024-01-21T22:53:00.611723+0000","last_clean":"2024-01-21T22:53:00.611723+0000","last_became_active":"2024-01-21T22:25:38.276512+0000","last_became_peered":"2024-01-21T22:25:38.276512+0000","last_unstale":"2024-01-21T22:53:00.611723+0000","last_undegraded":"2024-01-21T22:53:00.611723+0000","last_fullsized":"2024-01-21T22:53:00.611723+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'166","last_scrub_stamp":"2024-01-21T22:53:00.611621+0000","last_deep_scrub":"25'166","last_deep_scrub_stamp":"2024-01-21T22:53:00.611621+0000","last_clean_scrub_stamp":"2024-01-21T22:53:00.611621+0000","objects_scrubbed":0,"log_size":166,"log_dups_size":0,"ondisk_log_size":166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:38:05.023798+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":119,"num_write_kb":11355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"25'629","reported_seq":666,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:55.407868+0000","last_change":"2024-01-21T22:52:55.407896+0000","last_active":"2024-01-21T22:52:55.407868+0000","last_peered":"2024-01-21T22:52:55.407868+0000","last_clean":"2024-01-21T22:52:55.407868+0000","last_became_active":"2024-01-21T22:25:03.708101+0000","last_became_peered":"2024-01-21T22:25:03.708101+0000","last_unstale":"2024-01-21T22:52:55.407868+0000","last_undegraded":"2024-01-21T22:52:55.407868+0000","last_fullsized":"2024-01-21T22:52:55.407868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'629","last_scrub_stamp":"2024-01-21T22:52:55.407801+0000","last_deep_scrub":"25'629","last_deep_scrub_stamp":"2024-01-21T22:52:55.407801+0000","last_clean_scrub_stamp":"2024-01-21T22:52:55.407801+0000","objects_scrubbed":2,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:29:05.002686+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194374,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":625,"num_write_kb":4435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"25'198","reported_seq":241,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:53.936677+0000","last_change":"2024-01-21T22:52:53.936726+0000","last_active":"2024-01-21T22:52:53.936677+0000","last_peered":"2024-01-21T22:52:53.936677+0000","last_clean":"2024-01-21T22:52:53.936677+0000","last_became_active":"2024-01-21T22:25:04.877661+0000","last_became_peered":"2024-01-21T22:25:04.877661+0000","last_unstale":"2024-01-21T22:52:53.936677+0000","last_undegraded":"2024-01-21T22:52:53.936677+0000","last_fullsized":"2024-01-21T22:52:53.936677+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'198","last_scrub_stamp":"2024-01-21T22:52:53.936554+0000","last_deep_scrub":"25'198","last_deep_scrub_stamp":"2024-01-21T22:52:53.936554+0000","last_clean_scrub_stamp":"2024-01-21T22:52:53.936554+0000","objects_scrubbed":0,"log_size":198,"log_dups_size":0,"ondisk_log_size":198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:19:00.477454+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":145,"num_write_kb":7036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"25'2","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:54.517950+0000","last_change":"2024-01-21T22:52:54.517987+0000","last_active":"2024-01-21T22:52:54.517950+0000","last_peered":"2024-01-21T22:52:54.517950+0000","last_clean":"2024-01-21T22:52:54.517950+0000","last_became_active":"2024-01-21T22:25:03.721281+0000","last_became_peered":"2024-01-21T22:25:03.721281+0000","last_unstale":"2024-01-21T22:52:54.517950+0000","last_undegraded":"2024-01-21T22:52:54.517950+0000","last_fullsized":"2024-01-21T22:52:54.517950+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-01-21T22:52:54.517822+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-01-21T22:52:54.517822+0000","last_clean_scrub_stamp":"2024-01-21T22:52:54.517822+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:58:00.732731+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"25'173","reported_seq":226,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:06.451514+0000","last_change":"2024-01-21T22:53:06.451605+0000","last_active":"2024-01-21T22:53:06.451514+0000","last_peered":"2024-01-21T22:53:06.451514+0000","last_clean":"2024-01-21T22:53:06.451514+0000","last_became_active":"2024-01-21T22:25:04.880227+0000","last_became_peered":"2024-01-21T22:25:04.880227+0000","last_unstale":"2024-01-21T22:53:06.451514+0000","last_undegraded":"2024-01-21T22:53:06.451514+0000","last_fullsized":"2024-01-21T22:53:06.451514+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'173","last_scrub_stamp":"2024-01-21T22:53:06.451297+0000","last_deep_scrub":"25'173","last_deep_scrub_stamp":"2024-01-21T22:53:06.451297+0000","last_clean_scrub_stamp":"2024-01-21T22:53:06.451297+0000","objects_scrubbed":0,"log_size":173,"log_dups_size":0,"ondisk_log_size":173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:17:52.564284+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":4318,"num_write":118,"num_write_kb":4353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"25'119","reported_seq":193,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:48.910070+0000","last_change":"2024-01-21T22:52:48.910070+0000","last_active":"2024-01-21T22:52:48.910070+0000","last_peered":"2024-01-21T22:52:48.910070+0000","last_clean":"2024-01-21T22:52:48.910070+0000","last_became_active":"2024-01-21T22:24:52.967018+0000","last_became_peered":"2024-01-21T22:24:52.967018+0000","last_unstale":"2024-01-21T22:52:48.910070+0000","last_undegraded":"2024-01-21T22:52:48.910070+0000","last_fullsized":"2024-01-21T22:52:48.910070+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'119","last_scrub_stamp":"2024-01-21T22:52:48.909687+0000","last_deep_scrub":"25'119","last_deep_scrub_stamp":"2024-01-21T22:52:48.909687+0000","last_clean_scrub_stamp":"2024-01-21T22:52:48.909687+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:14:14.043092+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"25'6","reported_seq":44,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:00.538127+0000","last_change":"2024-01-21T22:53:00.538183+0000","last_active":"2024-01-21T22:53:00.538127+0000","last_peered":"2024-01-21T22:53:00.538127+0000","last_clean":"2024-01-21T22:53:00.538127+0000","last_became_active":"2024-01-21T22:25:03.707085+0000","last_became_peered":"2024-01-21T22:25:03.707085+0000","last_unstale":"2024-01-21T22:53:00.538127+0000","last_undegraded":"2024-01-21T22:53:00.538127+0000","last_fullsized":"2024-01-21T22:53:00.538127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'6","last_scrub_stamp":"2024-01-21T22:53:00.537931+0000","last_deep_scrub":"25'6","last_deep_scrub_stamp":"2024-01-21T22:53:00.537931+0000","last_clean_scrub_stamp":"2024-01-21T22:53:00.537931+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:20:57.912995+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"25'177","reported_seq":224,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:02.090444+0000","last_change":"2024-01-21T22:53:02.090466+0000","last_active":"2024-01-21T22:53:02.090444+0000","last_peered":"2024-01-21T22:53:02.090444+0000","last_clean":"2024-01-21T22:53:02.090444+0000","last_became_active":"2024-01-21T22:25:04.868247+0000","last_became_peered":"2024-01-21T22:25:04.868247+0000","last_unstale":"2024-01-21T22:53:02.090444+0000","last_undegraded":"2024-01-21T22:53:02.090444+0000","last_fullsized":"2024-01-21T22:53:02.090444+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'177","last_scrub_stamp":"2024-01-21T22:53:02.090374+0000","last_deep_scrub":"25'177","last_deep_scrub_stamp":"2024-01-21T22:53:02.090374+0000","last_clean_scrub_stamp":"2024-01-21T22:53:02.090374+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T10:41:05.521639+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":2540,"num_write":132,"num_write_kb":4885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"25'1295","reported_seq":1334,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:59.604568+0000","last_change":"2024-01-21T22:52:59.604568+0000","last_active":"2024-01-21T22:52:59.604568+0000","last_peered":"2024-01-21T22:52:59.604568+0000","last_clean":"2024-01-21T22:52:59.604568+0000","last_became_active":"2024-01-21T22:25:03.705785+0000","last_became_peered":"2024-01-21T22:25:03.705785+0000","last_unstale":"2024-01-21T22:52:59.604568+0000","last_undegraded":"2024-01-21T22:52:59.604568+0000","last_fullsized":"2024-01-21T22:52:59.604568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1295","last_scrub_stamp":"2024-01-21T22:52:59.604320+0000","last_deep_scrub":"25'1295","last_deep_scrub_stamp":"2024-01-21T22:52:59.604320+0000","last_clean_scrub_stamp":"2024-01-21T22:52:59.604320+0000","objects_scrubbed":2,"log_size":1295,"log_dups_size":0,"ondisk_log_size":1295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T10:10:43.483091+0000","scrub_duration":54,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1286,"num_write_kb":8869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"25'188","reported_seq":237,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:59.663102+0000","last_change":"2024-01-21T22:52:59.663136+0000","last_active":"2024-01-21T22:52:59.663102+0000","last_peered":"2024-01-21T22:52:59.663102+0000","last_clean":"2024-01-21T22:52:59.663102+0000","last_became_active":"2024-01-21T22:25:04.845274+0000","last_became_peered":"2024-01-21T22:25:04.845274+0000","last_unstale":"2024-01-21T22:52:59.663102+0000","last_undegraded":"2024-01-21T22:52:59.663102+0000","last_fullsized":"2024-01-21T22:52:59.663102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'188","last_scrub_stamp":"2024-01-21T22:52:59.663004+0000","last_deep_scrub":"25'188","last_deep_scrub_stamp":"2024-01-21T22:52:59.663004+0000","last_clean_scrub_stamp":"2024-01-21T22:52:59.663004+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:07:13.329609+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":3048,"num_write":151,"num_write_kb":10622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"25'756","reported_seq":1540,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:55.561845+0000","last_change":"2024-01-21T22:52:55.561869+0000","last_active":"2024-01-21T22:52:55.561845+0000","last_peered":"2024-01-21T22:52:55.561845+0000","last_clean":"2024-01-21T22:52:55.561845+0000","last_became_active":"2024-01-21T22:25:03.708894+0000","last_became_peered":"2024-01-21T22:25:03.708894+0000","last_unstale":"2024-01-21T22:52:55.561845+0000","last_undegraded":"2024-01-21T22:52:55.561845+0000","last_fullsized":"2024-01-21T22:52:55.561845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'756","last_scrub_stamp":"2024-01-21T22:52:55.561772+0000","last_deep_scrub":"25'756","last_deep_scrub_stamp":"2024-01-21T22:52:55.561772+0000","last_clean_scrub_stamp":"2024-01-21T22:52:55.561772+0000","objects_scrubbed":1,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:35:50.369682+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":610747,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":746,"num_read_kb":1116,"num_write":750,"num_write_kb":1119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"25'188","reported_seq":240,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:57.400366+0000","last_change":"2024-01-21T22:52:57.400366+0000","last_active":"2024-01-21T22:52:57.400366+0000","last_peered":"2024-01-21T22:52:57.400366+0000","last_clean":"2024-01-21T22:52:57.400366+0000","last_became_active":"2024-01-21T22:25:04.711694+0000","last_became_peered":"2024-01-21T22:25:04.711694+0000","last_unstale":"2024-01-21T22:52:57.400366+0000","last_undegraded":"2024-01-21T22:52:57.400366+0000","last_fullsized":"2024-01-21T22:52:57.400366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'188","last_scrub_stamp":"2024-01-21T22:52:57.400252+0000","last_deep_scrub":"25'188","last_deep_scrub_stamp":"2024-01-21T22:52:57.400252+0000","last_clean_scrub_stamp":"2024-01-21T22:52:57.400252+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:49:18.313031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3556,"num_write":135,"num_write_kb":5944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"25'1298","reported_seq":1336,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:57.663807+0000","last_change":"2024-01-21T22:52:57.663881+0000","last_active":"2024-01-21T22:52:57.663807+0000","last_peered":"2024-01-21T22:52:57.663807+0000","last_clean":"2024-01-21T22:52:57.663807+0000","last_became_active":"2024-01-21T22:25:03.703876+0000","last_became_peered":"2024-01-21T22:25:03.703876+0000","last_unstale":"2024-01-21T22:52:57.663807+0000","last_undegraded":"2024-01-21T22:52:57.663807+0000","last_fullsized":"2024-01-21T22:52:57.663807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1298","last_scrub_stamp":"2024-01-21T22:52:57.663606+0000","last_deep_scrub":"25'1298","last_deep_scrub_stamp":"2024-01-21T22:52:57.663606+0000","last_clean_scrub_stamp":"2024-01-21T22:52:57.663606+0000","objects_scrubbed":2,"log_size":1298,"log_dups_size":0,"ondisk_log_size":1298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:20:12.580938+0000","scrub_duration":53,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1291,"num_write_kb":8882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"25'230","reported_seq":276,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:59.967303+0000","last_change":"2024-01-21T22:52:59.967365+0000","last_active":"2024-01-21T22:52:59.967303+0000","last_peered":"2024-01-21T22:52:59.967303+0000","last_clean":"2024-01-21T22:52:59.967303+0000","last_became_active":"2024-01-21T22:25:04.884162+0000","last_became_peered":"2024-01-21T22:25:04.884162+0000","last_unstale":"2024-01-21T22:52:59.967303+0000","last_undegraded":"2024-01-21T22:52:59.967303+0000","last_fullsized":"2024-01-21T22:52:59.967303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'230","last_scrub_stamp":"2024-01-21T22:52:59.967119+0000","last_deep_scrub":"25'230","last_deep_scrub_stamp":"2024-01-21T22:52:59.967119+0000","last_clean_scrub_stamp":"2024-01-21T22:52:59.967119+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T06:00:09.786231+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":180,"num_write_kb":15842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"25'4","reported_seq":42,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:54.357912+0000","last_change":"2024-01-21T22:52:54.357937+0000","last_active":"2024-01-21T22:52:54.357912+0000","last_peered":"2024-01-21T22:52:54.357912+0000","last_clean":"2024-01-21T22:52:54.357912+0000","last_became_active":"2024-01-21T22:25:03.708036+0000","last_became_peered":"2024-01-21T22:25:03.708036+0000","last_unstale":"2024-01-21T22:52:54.357912+0000","last_undegraded":"2024-01-21T22:52:54.357912+0000","last_fullsized":"2024-01-21T22:52:54.357912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'4","last_scrub_stamp":"2024-01-21T22:52:54.357828+0000","last_deep_scrub":"25'4","last_deep_scrub_stamp":"2024-01-21T22:52:54.357828+0000","last_clean_scrub_stamp":"2024-01-21T22:52:54.357828+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:52:06.576829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"25'188","reported_seq":241,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:52.078481+0000","last_change":"2024-01-21T22:52:52.078542+0000","last_active":"2024-01-21T22:52:52.078481+0000","last_peered":"2024-01-21T22:52:52.078481+0000","last_clean":"2024-01-21T22:52:52.078481+0000","last_became_active":"2024-01-21T22:25:04.844008+0000","last_became_peered":"2024-01-21T22:25:04.844008+0000","last_unstale":"2024-01-21T22:52:52.078481+0000","last_undegraded":"2024-01-21T22:52:52.078481+0000","last_fullsized":"2024-01-21T22:52:52.078481+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'188","last_scrub_stamp":"2024-01-21T22:52:52.078302+0000","last_deep_scrub":"25'188","last_deep_scrub_stamp":"2024-01-21T22:52:52.078302+0000","last_clean_scrub_stamp":"2024-01-21T22:52:52.078302+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:05:20.309821+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":4064,"num_write":142,"num_write_kb":4003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"25'3","reported_seq":41,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:55.610922+0000","last_change":"2024-01-21T22:52:55.610984+0000","last_active":"2024-01-21T22:52:55.610922+0000","last_peered":"2024-01-21T22:52:55.610922+0000","last_clean":"2024-01-21T22:52:55.610922+0000","last_became_active":"2024-01-21T22:25:03.698443+0000","last_became_peered":"2024-01-21T22:25:03.698443+0000","last_unstale":"2024-01-21T22:52:55.610922+0000","last_undegraded":"2024-01-21T22:52:55.610922+0000","last_fullsized":"2024-01-21T22:52:55.610922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'3","last_scrub_stamp":"2024-01-21T22:52:55.610738+0000","last_deep_scrub":"25'3","last_deep_scrub_stamp":"2024-01-21T22:52:55.610738+0000","last_clean_scrub_stamp":"2024-01-21T22:52:55.610738+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:38:20.687217+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"25'213","reported_seq":257,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:57.582658+0000","last_change":"2024-01-21T22:52:57.582681+0000","last_active":"2024-01-21T22:52:57.582658+0000","last_peered":"2024-01-21T22:52:57.582658+0000","last_clean":"2024-01-21T22:52:57.582658+0000","last_became_active":"2024-01-21T22:25:04.842479+0000","last_became_peered":"2024-01-21T22:25:04.842479+0000","last_unstale":"2024-01-21T22:52:57.582658+0000","last_undegraded":"2024-01-21T22:52:57.582658+0000","last_fullsized":"2024-01-21T22:52:57.582658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'213","last_scrub_stamp":"2024-01-21T22:52:57.582573+0000","last_deep_scrub":"25'213","last_deep_scrub_stamp":"2024-01-21T22:52:57.582573+0000","last_clean_scrub_stamp":"2024-01-21T22:52:57.582573+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T10:35:01.272418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":156,"num_write_kb":5205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"25'1421","reported_seq":1459,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:55.265742+0000","last_change":"2024-01-21T22:52:55.265742+0000","last_active":"2024-01-21T22:52:55.265742+0000","last_peered":"2024-01-21T22:52:55.265742+0000","last_clean":"2024-01-21T22:52:55.265742+0000","last_became_active":"2024-01-21T22:25:03.720067+0000","last_became_peered":"2024-01-21T22:25:03.720067+0000","last_unstale":"2024-01-21T22:52:55.265742+0000","last_undegraded":"2024-01-21T22:52:55.265742+0000","last_fullsized":"2024-01-21T22:52:55.265742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1421","last_scrub_stamp":"2024-01-21T22:52:55.265581+0000","last_deep_scrub":"25'1421","last_deep_scrub_stamp":"2024-01-21T22:52:55.265581+0000","last_clean_scrub_stamp":"2024-01-21T22:52:55.265581+0000","objects_scrubbed":1,"log_size":1421,"log_dups_size":0,"ondisk_log_size":1421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:17:23.614504+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3488,"num_write_kb":1512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"25'225","reported_seq":284,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:59.139088+0000","last_change":"2024-01-21T22:52:59.139111+0000","last_active":"2024-01-21T22:52:59.139088+0000","last_peered":"2024-01-21T22:52:59.139088+0000","last_clean":"2024-01-21T22:52:59.139088+0000","last_became_active":"2024-01-21T22:25:04.859270+0000","last_became_peered":"2024-01-21T22:25:04.859270+0000","last_unstale":"2024-01-21T22:52:59.139088+0000","last_undegraded":"2024-01-21T22:52:59.139088+0000","last_fullsized":"2024-01-21T22:52:59.139088+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'225","last_scrub_stamp":"2024-01-21T22:52:59.139010+0000","last_deep_scrub":"25'225","last_deep_scrub_stamp":"2024-01-21T22:52:59.139010+0000","last_clean_scrub_stamp":"2024-01-21T22:52:59.139010+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:19:39.340177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":5842,"num_write":177,"num_write_kb":7531,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"25'8","reported_seq":46,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:54.581660+0000","last_change":"2024-01-21T22:52:54.581660+0000","last_active":"2024-01-21T22:52:54.581660+0000","last_peered":"2024-01-21T22:52:54.581660+0000","last_clean":"2024-01-21T22:52:54.581660+0000","last_became_active":"2024-01-21T22:25:03.721275+0000","last_became_peered":"2024-01-21T22:25:03.721275+0000","last_unstale":"2024-01-21T22:52:54.581660+0000","last_undegraded":"2024-01-21T22:52:54.581660+0000","last_fullsized":"2024-01-21T22:52:54.581660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'8","last_scrub_stamp":"2024-01-21T22:52:54.581508+0000","last_deep_scrub":"25'8","last_deep_scrub_stamp":"2024-01-21T22:52:54.581508+0000","last_clean_scrub_stamp":"2024-01-21T22:52:54.581508+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T06:27:57.295530+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"25'182","reported_seq":231,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:58.155468+0000","last_change":"2024-01-21T22:52:58.155511+0000","last_active":"2024-01-21T22:52:58.155468+0000","last_peered":"2024-01-21T22:52:58.155468+0000","last_clean":"2024-01-21T22:52:58.155468+0000","last_became_active":"2024-01-21T22:25:04.858586+0000","last_became_peered":"2024-01-21T22:25:04.858586+0000","last_unstale":"2024-01-21T22:52:58.155468+0000","last_undegraded":"2024-01-21T22:52:58.155468+0000","last_fullsized":"2024-01-21T22:52:58.155468+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'182","last_scrub_stamp":"2024-01-21T22:52:58.155389+0000","last_deep_scrub":"25'182","last_deep_scrub_stamp":"2024-01-21T22:52:58.155389+0000","last_clean_scrub_stamp":"2024-01-21T22:52:58.155389+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:29:07.750274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":132,"num_write_kb":7745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"25'672","reported_seq":710,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:49.868747+0000","last_change":"2024-01-21T22:52:49.868747+0000","last_active":"2024-01-21T22:52:49.868747+0000","last_peered":"2024-01-21T22:52:49.868747+0000","last_clean":"2024-01-21T22:52:49.868747+0000","last_became_active":"2024-01-21T22:25:03.718171+0000","last_became_peered":"2024-01-21T22:25:03.718171+0000","last_unstale":"2024-01-21T22:52:49.868747+0000","last_undegraded":"2024-01-21T22:52:49.868747+0000","last_fullsized":"2024-01-21T22:52:49.868747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'672","last_scrub_stamp":"2024-01-21T22:52:49.868570+0000","last_deep_scrub":"25'672","last_deep_scrub_stamp":"2024-01-21T22:52:49.868570+0000","last_clean_scrub_stamp":"2024-01-21T22:52:49.868570+0000","objects_scrubbed":1,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:50:05.726919+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":666,"num_write_kb":4450,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"25'192","reported_seq":237,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:52.570424+0000","last_change":"2024-01-21T22:52:52.570449+0000","last_active":"2024-01-21T22:52:52.570424+0000","last_peered":"2024-01-21T22:52:52.570424+0000","last_clean":"2024-01-21T22:52:52.570424+0000","last_became_active":"2024-01-21T22:25:04.875368+0000","last_became_peered":"2024-01-21T22:25:04.875368+0000","last_unstale":"2024-01-21T22:52:52.570424+0000","last_undegraded":"2024-01-21T22:52:52.570424+0000","last_fullsized":"2024-01-21T22:52:52.570424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'192","last_scrub_stamp":"2024-01-21T22:52:52.570345+0000","last_deep_scrub":"25'192","last_deep_scrub_stamp":"2024-01-21T22:52:52.570345+0000","last_clean_scrub_stamp":"2024-01-21T22:52:52.570345+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:14:45.162994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":2286,"num_write":138,"num_write_kb":6957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"25'4","reported_seq":42,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:56.215793+0000","last_change":"2024-01-21T22:52:56.215793+0000","last_active":"2024-01-21T22:52:56.215793+0000","last_peered":"2024-01-21T22:52:56.215793+0000","last_clean":"2024-01-21T22:52:56.215793+0000","last_became_active":"2024-01-21T22:25:03.722525+0000","last_became_peered":"2024-01-21T22:25:03.722525+0000","last_unstale":"2024-01-21T22:52:56.215793+0000","last_undegraded":"2024-01-21T22:52:56.215793+0000","last_fullsized":"2024-01-21T22:52:56.215793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'4","last_scrub_stamp":"2024-01-21T22:52:56.215663+0000","last_deep_scrub":"25'4","last_deep_scrub_stamp":"2024-01-21T22:52:56.215663+0000","last_clean_scrub_stamp":"2024-01-21T22:52:56.215663+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T04:07:09.882268+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"25'178","reported_seq":227,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:57.190547+0000","last_change":"2024-01-21T22:52:57.190588+0000","last_active":"2024-01-21T22:52:57.190547+0000","last_peered":"2024-01-21T22:52:57.190547+0000","last_clean":"2024-01-21T22:52:57.190547+0000","last_became_active":"2024-01-21T22:25:04.857205+0000","last_became_peered":"2024-01-21T22:25:04.857205+0000","last_unstale":"2024-01-21T22:52:57.190547+0000","last_undegraded":"2024-01-21T22:52:57.190547+0000","last_fullsized":"2024-01-21T22:52:57.190547+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'178","last_scrub_stamp":"2024-01-21T22:52:57.190448+0000","last_deep_scrub":"25'178","last_deep_scrub_stamp":"2024-01-21T22:52:57.190448+0000","last_clean_scrub_stamp":"2024-01-21T22:52:57.190448+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:26:42.486813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":3302,"num_write":126,"num_write_kb":4840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"25'2","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:53.563879+0000","last_change":"2024-01-21T22:52:53.563922+0000","last_active":"2024-01-21T22:52:53.563879+0000","last_peered":"2024-01-21T22:52:53.563879+0000","last_clean":"2024-01-21T22:52:53.563879+0000","last_became_active":"2024-01-21T22:25:03.708318+0000","last_became_peered":"2024-01-21T22:25:03.708318+0000","last_unstale":"2024-01-21T22:52:53.563879+0000","last_undegraded":"2024-01-21T22:52:53.563879+0000","last_fullsized":"2024-01-21T22:52:53.563879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-01-21T22:52:53.563796+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-01-21T22:52:53.563796+0000","last_clean_scrub_stamp":"2024-01-21T22:52:53.563796+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:25:01.093202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"25'222","reported_seq":291,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:59.642050+0000","last_change":"2024-01-21T22:52:59.642072+0000","last_active":"2024-01-21T22:52:59.642050+0000","last_peered":"2024-01-21T22:52:59.642050+0000","last_clean":"2024-01-21T22:52:59.642050+0000","last_became_active":"2024-01-21T22:25:04.842815+0000","last_became_peered":"2024-01-21T22:25:04.842815+0000","last_unstale":"2024-01-21T22:52:59.642050+0000","last_undegraded":"2024-01-21T22:52:59.642050+0000","last_fullsized":"2024-01-21T22:52:59.642050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'222","last_scrub_stamp":"2024-01-21T22:52:59.641984+0000","last_deep_scrub":"25'222","last_deep_scrub_stamp":"2024-01-21T22:52:59.641984+0000","last_clean_scrub_stamp":"2024-01-21T22:52:59.641984+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T10:12:28.070400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":8382,"num_write":174,"num_write_kb":9170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"25'766","reported_seq":803,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:01.523349+0000","last_change":"2024-01-21T22:53:01.523420+0000","last_active":"2024-01-21T22:53:01.523349+0000","last_peered":"2024-01-21T22:53:01.523349+0000","last_clean":"2024-01-21T22:53:01.523349+0000","last_became_active":"2024-01-21T22:25:03.710262+0000","last_became_peered":"2024-01-21T22:25:03.710262+0000","last_unstale":"2024-01-21T22:53:01.523349+0000","last_undegraded":"2024-01-21T22:53:01.523349+0000","last_fullsized":"2024-01-21T22:53:01.523349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'766","last_scrub_stamp":"2024-01-21T22:53:01.523153+0000","last_deep_scrub":"25'766","last_deep_scrub_stamp":"2024-01-21T22:53:01.523153+0000","last_clean_scrub_stamp":"2024-01-21T22:53:01.523153+0000","objects_scrubbed":2,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T07:49:03.554794+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":756,"num_write_kb":4574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"25'193","reported_seq":243,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:50.594522+0000","last_change":"2024-01-21T22:52:50.594559+0000","last_active":"2024-01-21T22:52:50.594522+0000","last_peered":"2024-01-21T22:52:50.594522+0000","last_clean":"2024-01-21T22:52:50.594522+0000","last_became_active":"2024-01-21T22:25:04.875619+0000","last_became_peered":"2024-01-21T22:25:04.875619+0000","last_unstale":"2024-01-21T22:52:50.594522+0000","last_undegraded":"2024-01-21T22:52:50.594522+0000","last_fullsized":"2024-01-21T22:52:50.594522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'193","last_scrub_stamp":"2024-01-21T22:52:50.594392+0000","last_deep_scrub":"25'193","last_deep_scrub_stamp":"2024-01-21T22:52:50.594392+0000","last_clean_scrub_stamp":"2024-01-21T22:52:50.594392+0000","objects_scrubbed":0,"log_size":193,"log_dups_size":0,"ondisk_log_size":193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T05:18:57.135293+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":3556,"num_write":140,"num_write_kb":4746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"25'155","reported_seq":197,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:10.409237+0000","last_change":"2024-01-21T22:53:10.409297+0000","last_active":"2024-01-21T22:53:10.409237+0000","last_peered":"2024-01-21T22:53:10.409237+0000","last_clean":"2024-01-21T22:53:10.409237+0000","last_became_active":"2024-01-21T22:25:04.890515+0000","last_became_peered":"2024-01-21T22:25:04.890515+0000","last_unstale":"2024-01-21T22:53:10.409237+0000","last_undegraded":"2024-01-21T22:53:10.409237+0000","last_fullsized":"2024-01-21T22:53:10.409237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'155","last_scrub_stamp":"2024-01-21T22:53:10.409036+0000","last_deep_scrub":"25'155","last_deep_scrub_stamp":"2024-01-21T22:53:10.409036+0000","last_clean_scrub_stamp":"2024-01-21T22:53:10.409036+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T02:00:29.917097+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":1778,"num_write":88,"num_write_kb":3797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"25'206","reported_seq":242,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:51.899411+0000","last_change":"2024-01-21T22:52:51.899452+0000","last_active":"2024-01-21T22:52:51.899411+0000","last_peered":"2024-01-21T22:52:51.899411+0000","last_clean":"2024-01-21T22:52:51.899411+0000","last_became_active":"2024-01-21T22:25:03.718163+0000","last_became_peered":"2024-01-21T22:25:03.718163+0000","last_unstale":"2024-01-21T22:52:51.899411+0000","last_undegraded":"2024-01-21T22:52:51.899411+0000","last_fullsized":"2024-01-21T22:52:51.899411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'206","last_scrub_stamp":"2024-01-21T22:52:51.899295+0000","last_deep_scrub":"25'206","last_deep_scrub_stamp":"2024-01-21T22:52:51.899295+0000","last_clean_scrub_stamp":"2024-01-21T22:52:51.899295+0000","objects_scrubbed":2,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T10:51:29.160818+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1916505,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":203,"num_write_kb":1995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"25'212","reported_seq":253,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:54.972466+0000","last_change":"2024-01-21T22:52:54.972527+0000","last_active":"2024-01-21T22:52:54.972466+0000","last_peered":"2024-01-21T22:52:54.972466+0000","last_clean":"2024-01-21T22:52:54.972466+0000","last_became_active":"2024-01-21T22:25:04.888910+0000","last_became_peered":"2024-01-21T22:25:04.888910+0000","last_unstale":"2024-01-21T22:52:54.972466+0000","last_undegraded":"2024-01-21T22:52:54.972466+0000","last_fullsized":"2024-01-21T22:52:54.972466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'212","last_scrub_stamp":"2024-01-21T22:52:54.972300+0000","last_deep_scrub":"25'212","last_deep_scrub_stamp":"2024-01-21T22:52:54.972300+0000","last_clean_scrub_stamp":"2024-01-21T22:52:54.972300+0000","objects_scrubbed":0,"log_size":212,"log_dups_size":0,"ondisk_log_size":212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:30:33.729528+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":1524,"num_write":155,"num_write_kb":13062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"25'1487","reported_seq":1523,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:50.946510+0000","last_change":"2024-01-21T22:52:50.946563+0000","last_active":"2024-01-21T22:52:50.946510+0000","last_peered":"2024-01-21T22:52:50.946510+0000","last_clean":"2024-01-21T22:52:50.946510+0000","last_became_active":"2024-01-21T22:25:03.718649+0000","last_became_peered":"2024-01-21T22:25:03.718649+0000","last_unstale":"2024-01-21T22:52:50.946510+0000","last_undegraded":"2024-01-21T22:52:50.946510+0000","last_fullsized":"2024-01-21T22:52:50.946510+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'1487","last_scrub_stamp":"2024-01-21T22:52:50.946367+0000","last_deep_scrub":"25'1487","last_deep_scrub_stamp":"2024-01-21T22:52:50.946367+0000","last_clean_scrub_stamp":"2024-01-21T22:52:50.946367+0000","objects_scrubbed":3,"log_size":1487,"log_dups_size":0,"ondisk_log_size":1487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T08:49:11.859146+0000","scrub_duration":64,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1478,"num_write_kb":9119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"25'210","reported_seq":274,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:59.395656+0000","last_change":"2024-01-21T22:52:59.395678+0000","last_active":"2024-01-21T22:52:59.395656+0000","last_peered":"2024-01-21T22:52:59.395656+0000","last_clean":"2024-01-21T22:52:59.395656+0000","last_became_active":"2024-01-21T22:25:04.714720+0000","last_became_peered":"2024-01-21T22:25:04.714720+0000","last_unstale":"2024-01-21T22:52:59.395656+0000","last_undegraded":"2024-01-21T22:52:59.395656+0000","last_fullsized":"2024-01-21T22:52:59.395656+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'210","last_scrub_stamp":"2024-01-21T22:52:59.395584+0000","last_deep_scrub":"25'210","last_deep_scrub_stamp":"2024-01-21T22:52:59.395584+0000","last_clean_scrub_stamp":"2024-01-21T22:52:59.395584+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T03:07:51.631937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":27,"num_read_kb":6858,"num_write":147,"num_write_kb":17332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"25'748","reported_seq":787,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:53.366562+0000","last_change":"2024-01-21T22:52:53.366562+0000","last_active":"2024-01-21T22:52:53.366562+0000","last_peered":"2024-01-21T22:52:53.366562+0000","last_clean":"2024-01-21T22:52:53.366562+0000","last_became_active":"2024-01-21T22:25:03.703020+0000","last_became_peered":"2024-01-21T22:25:03.703020+0000","last_unstale":"2024-01-21T22:52:53.366562+0000","last_undegraded":"2024-01-21T22:52:53.366562+0000","last_fullsized":"2024-01-21T22:52:53.366562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'748","last_scrub_stamp":"2024-01-21T22:52:53.366456+0000","last_deep_scrub":"25'748","last_deep_scrub_stamp":"2024-01-21T22:52:53.366456+0000","last_clean_scrub_stamp":"2024-01-21T22:52:53.366456+0000","objects_scrubbed":1,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T01:43:23.832616+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":743,"num_write_kb":4567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"25'216","reported_seq":276,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:53:00.402887+0000","last_change":"2024-01-21T22:53:00.402887+0000","last_active":"2024-01-21T22:53:00.402887+0000","last_peered":"2024-01-21T22:53:00.402887+0000","last_clean":"2024-01-21T22:53:00.402887+0000","last_became_active":"2024-01-21T22:25:04.715652+0000","last_became_peered":"2024-01-21T22:25:04.715652+0000","last_unstale":"2024-01-21T22:53:00.402887+0000","last_undegraded":"2024-01-21T22:53:00.402887+0000","last_fullsized":"2024-01-21T22:53:00.402887+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'216","last_scrub_stamp":"2024-01-21T22:53:00.402783+0000","last_deep_scrub":"25'216","last_deep_scrub_stamp":"2024-01-21T22:53:00.402783+0000","last_clean_scrub_stamp":"2024-01-21T22:53:00.402783+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:23:45.193691+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":5588,"num_write":182,"num_write_kb":13243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"25'765","reported_seq":803,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-01-21T22:52:52.281253+0000","last_change":"2024-01-21T22:52:52.281313+0000","last_active":"2024-01-21T22:52:52.281253+0000","last_peered":"2024-01-21T22:52:52.281253+0000","last_clean":"2024-01-21T22:52:52.281253+0000","last_became_active":"2024-01-21T22:25:03.702087+0000","last_became_peered":"2024-01-21T22:25:03.702087+0000","last_unstale":"2024-01-21T22:52:52.281253+0000","last_undegraded":"2024-01-21T22:52:52.281253+0000","last_fullsized":"2024-01-21T22:52:52.281253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'765","last_scrub_stamp":"2024-01-21T22:52:52.281140+0000","last_deep_scrub":"25'765","last_deep_scrub_stamp":"2024-01-21T22:52:52.281140+0000","last_clean_scrub_stamp":"2024-01-21T22:52:52.281140+0000","objects_scrubbed":3,"log_size":765,"log_dups_size":0,"ondisk_log_size":765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-01-23T00:37:56.693317+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194886,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":760,"num_write_kb":4560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":833,"num_read_kb":211582,"num_write":8969,"num_write_kb":506711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12284,"ondisk_log_size":12284,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":82220498,"num_objects":42,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":0,"num_read":746,"num_read_kb":1116,"num_write":17695,"num_write_kb":90278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":164503552,"data_stored":164440996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":15789,"ondisk_log_size":15789,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542490,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62376,"kb_used_data":13152,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94309464,"statfs":{"total":96636764160,"available":96572891136,"internally_reserved":0,"allocated":13467648,"data_stored":13290724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.66800000000000004,"15min":0.68300000000000005},"min":{"1min":0.47699999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.872,"5min":1.0189999999999999,"15min":1.0640000000000001},"last":0.626},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70599999999999996,"15min":0.69699999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.024,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.875}]},{"osd":1,"last update":"Sun Jan 21 22:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.70299999999999996,"15min":0.70899999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.34000000000000002},"max":{"1min":0.98099999999999998,"5min":1.159,"15min":1.4179999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.999,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.998}]},{"osd":2,"last update":"Sun Jan 21 22:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.39300000000000002},"max":{"1min":1.002,"5min":1.0069999999999999,"15min":1.052},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.72199999999999998,"15min":0.72299999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.38},"max":{"1min":1.0409999999999999,"5min":1.1140000000000001,"15min":1.1799999999999999},"last":0.66500000000000004}]},{"osd":3,"last update":"Sun Jan 21 22:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.69299999999999995,"15min":0.71199999999999997},"min":{"1min":0.45900000000000002,"5min":0.42699999999999999,"15min":0.42099999999999999},"max":{"1min":0.89900000000000002,"5min":1.093,"15min":1.1539999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.95799999999999996,"5min":1.117,"15min":1.181},"last":0.63700000000000001}]},{"osd":4,"last update":"Sun Jan 21 22:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35099999999999998,"15min":0.34499999999999997},"min":{"1min":0.25,"5min":0.245,"15min":0.22700000000000001},"max":{"1min":0.505,"5min":0.57099999999999995,"15min":0.58299999999999996},"last":0.37},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.35699999999999998,"15min":0.34799999999999998},"min":{"1min":0.27300000000000002,"5min":0.248,"15min":0.23000000000000001},"max":{"1min":0.56499999999999995,"5min":0.66700000000000004,"15min":0.68100000000000005},"last":0.35699999999999998}]},{"osd":5,"last update":"Sun Jan 21 22:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34499999999999997,"15min":0.34300000000000003},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.46999999999999997,"5min":0.50900000000000001,"15min":0.50900000000000001},"last":0.312},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.373},"min":{"1min":0.29099999999999998,"5min":0.26700000000000002,"15min":0.249},"max":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.624},"last":0.40100000000000002}]},{"osd":6,"last update":"Sun Jan 21 22:52:51 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40100000000000002,"15min":0.39600000000000002},"min":{"1min":0.32500000000000001,"5min":0.312,"15min":0.27800000000000002},"max":{"1min":0.55000000000000004,"5min":0.57199999999999995,"15min":0.67300000000000004},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.38800000000000001,"15min":0.38500000000000001},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.26400000000000001},"max":{"1min":0.55000000000000004,"5min":0.60699999999999998,"15min":0.63700000000000001},"last":0.38900000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542490,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43720,"kb_used_data":12544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328120,"statfs":{"total":96636764160,"available":96591994880,"internally_reserved":0,"allocated":12845056,"data_stored":12679853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69299999999999995,"15min":0.70099999999999996},"min":{"1min":0.497,"5min":0.48299999999999998,"15min":0.45600000000000002},"max":{"1min":0.89200000000000002,"5min":1.0189999999999999,"15min":3.3380000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.54100000000000004,"5min":0.495,"15min":0.44600000000000001},"max":{"1min":0.93799999999999994,"5min":1.038,"15min":1.0669999999999999},"last":0.81899999999999995}]},{"osd":1,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72099999999999997,"15min":0.70699999999999996},"min":{"1min":0.46500000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0529999999999999,"5min":1.0860000000000001,"15min":2.9860000000000002},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72299999999999998,"15min":0.72099999999999997},"min":{"1min":0.52900000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.1020000000000001},"last":0.85099999999999998}]},{"osd":2,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.49299999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.88200000000000001,"5min":1.1220000000000001,"15min":7.5439999999999996},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71999999999999997,"15min":0.73199999999999998},"min":{"1min":0.59499999999999997,"5min":0.56200000000000006,"15min":0.495},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.86099999999999999}]},{"osd":3,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.72399999999999998,"15min":0.70499999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.42599999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73099999999999998,"15min":0.78100000000000003},"min":{"1min":0.56399999999999995,"5min":0.52700000000000002,"15min":0.39400000000000002},"max":{"1min":0.90100000000000002,"5min":1.075,"15min":8.4689999999999994},"last":0.66800000000000004}]},{"osd":4,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34300000000000003,"15min":0.33800000000000002},"min":{"1min":0.26000000000000001,"5min":0.245,"15min":0.222},"max":{"1min":0.45200000000000001,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.35599999999999998,"15min":0.35199999999999998},"min":{"1min":0.24199999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.45100000000000001,"5min":0.45600000000000002,"15min":0.91900000000000004},"last":0.51300000000000001}]},{"osd":5,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35199999999999998,"15min":0.35399999999999998},"min":{"1min":0.30199999999999999,"5min":0.26400000000000001,"15min":0.26100000000000001},"max":{"1min":0.40200000000000002,"5min":0.53800000000000003,"15min":0.61299999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.377,"15min":0.36799999999999999},"min":{"1min":0.29599999999999999,"5min":0.26500000000000001,"15min":0.26300000000000001},"max":{"1min":0.45300000000000001,"5min":0.46800000000000003,"15min":0.52200000000000002},"last":0.45900000000000002}]},{"osd":7,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.38},"min":{"1min":0.29799999999999999,"5min":0.27100000000000002,"15min":0.26600000000000001},"max":{"1min":0.50800000000000001,"5min":0.51100000000000001,"15min":0.96099999999999997},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.39200000000000002,"15min":0.39100000000000001},"min":{"1min":0.28899999999999998,"5min":0.28799999999999998,"15min":0.27300000000000002},"max":{"1min":0.45200000000000001,"5min":0.54900000000000004,"15min":0.98699999999999999},"last":0.49299999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542490,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":101612,"kb_used_data":26020,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94270228,"statfs":{"total":96636764160,"available":96532713472,"internally_reserved":0,"allocated":26644480,"data_stored":26463972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.67700000000000005,"15min":0.68899999999999995},"min":{"1min":0.43099999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96999999999999997,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.629},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.69599999999999995,"15min":0.69899999999999995},"min":{"1min":0.47599999999999998,"5min":0.432,"15min":0.41499999999999998},"max":{"1min":0.97799999999999998,"5min":1.0149999999999999,"15min":1.1020000000000001},"last":0.80300000000000005}]},{"osd":1,"last update":"Sun Jan 21 22:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68200000000000005,"15min":0.69099999999999995},"min":{"1min":0.42799999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":1.1950000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.72199999999999998,"15min":0.72499999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.40999999999999998},"max":{"1min":0.85999999999999999,"5min":1.111,"15min":1.111},"last":0.77300000000000002}]},{"osd":2,"last update":"Sun Jan 21 22:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.70099999999999996,"15min":0.71899999999999997},"min":{"1min":0.51400000000000001,"5min":0.48199999999999998,"15min":0.434},"max":{"1min":0.70399999999999996,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70599999999999996,"15min":0.71299999999999997},"min":{"1min":0.48399999999999999,"5min":0.437,"15min":0.437},"max":{"1min":0.98099999999999998,"5min":1.143,"15min":1.149},"last":0.94499999999999995}]},{"osd":3,"last update":"Sun Jan 21 22:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70199999999999996,"15min":0.72799999999999998},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.1419999999999999,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.754,"15min":0.72299999999999998},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.373},"max":{"1min":0.95099999999999996,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.79000000000000004}]},{"osd":4,"last update":"Sun Jan 21 22:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.33500000000000002,"15min":0.33600000000000002},"min":{"1min":0.29099999999999998,"5min":0.23599999999999999,"15min":0.22800000000000001},"max":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.34499999999999997,"15min":0.35099999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.25700000000000001},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.32300000000000001}]},{"osd":6,"last update":"Sun Jan 21 22:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.35199999999999998,"15min":0.35299999999999998},"min":{"1min":0.29999999999999999,"5min":0.26300000000000001,"15min":0.26000000000000001},"max":{"1min":0.42099999999999999,"5min":0.48899999999999999,"15min":0.51800000000000002},"last":0.309},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36599999999999999,"15min":0.36699999999999999},"min":{"1min":0.28100000000000003,"5min":0.27200000000000002,"15min":0.25600000000000001},"max":{"1min":0.442,"5min":0.48099999999999998,"15min":0.51100000000000001},"last":0.33900000000000002}]},{"osd":7,"last update":"Sun Jan 21 22:52:46 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.38300000000000001,"15min":0.38400000000000001},"min":{"1min":0.316,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38700000000000001,"15min":0.38500000000000001},"min":{"1min":0.311,"5min":0.27600000000000002,"15min":0.27100000000000002},"max":{"1min":0.51400000000000001,"5min":0.54100000000000004,"15min":0.57099999999999995},"last":0.35899999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542490,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65944,"kb_used_data":12560,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94305896,"statfs":{"total":96636764160,"available":96569237504,"internally_reserved":0,"allocated":12861440,"data_stored":12681197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.70299999999999996,"15min":0.68999999999999995},"min":{"1min":0.51100000000000001,"5min":0.46200000000000002,"15min":0.38800000000000001},"max":{"1min":0.97299999999999998,"5min":1.05,"15min":1.0880000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.69399999999999995,"15min":0.71499999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.41099999999999998},"max":{"1min":0.998,"5min":1.008,"15min":2.9649999999999999},"last":0.58699999999999997}]},{"osd":1,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.70599999999999996,"15min":0.71899999999999997},"min":{"1min":0.45800000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.9410000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.70199999999999996,"15min":0.71399999999999997},"min":{"1min":0.44,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.004,"5min":1.004,"15min":1.615},"last":0.60099999999999998}]},{"osd":2,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70499999999999996,"15min":0.71499999999999997},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.95799999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.73499999999999999,"15min":0.71799999999999997},"min":{"1min":0.54700000000000004,"5min":0.505,"15min":0.40400000000000003},"max":{"1min":1.05,"5min":1.05,"15min":1.4710000000000001},"last":0.61699999999999999}]},{"osd":3,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69899999999999995,"15min":0.69799999999999995},"min":{"1min":0.55400000000000005,"5min":0.48499999999999999,"15min":0.43099999999999999},"max":{"1min":0.97499999999999998,"5min":1.0640000000000001,"15min":1.0720000000000001},"last":0.629},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.72699999999999998,"15min":0.745},"min":{"1min":0.53800000000000003,"5min":0.52300000000000002,"15min":0.50900000000000001},"max":{"1min":0.92700000000000005,"5min":1.0600000000000001,"15min":1.1160000000000001},"last":0.83499999999999996}]},{"osd":5,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34499999999999997,"15min":0.34200000000000003},"min":{"1min":0.251,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.432,"5min":0.48199999999999998,"15min":0.48199999999999998},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.35099999999999998,"15min":0.34799999999999998},"min":{"1min":0.28000000000000003,"5min":0.252,"15min":0.249},"max":{"1min":0.42299999999999999,"5min":0.443,"15min":0.49399999999999999},"last":0.33600000000000002}]},{"osd":6,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.34999999999999998,"15min":0.34699999999999998},"min":{"1min":0.28100000000000003,"5min":0.24299999999999999,"15min":0.23599999999999999},"max":{"1min":0.42099999999999999,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.378,"15min":0.374},"min":{"1min":0.29199999999999998,"5min":0.27700000000000002,"15min":0.26400000000000001},"max":{"1min":0.46800000000000003,"5min":0.51500000000000001,"15min":0.54000000000000004},"last":0.39200000000000002}]},{"osd":7,"last update":"Sun Jan 21 22:52:34 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38500000000000001,"15min":0.375},"min":{"1min":0.27600000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.52000000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39600000000000002,"15min":0.39000000000000001},"min":{"1min":0.317,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.47699999999999998,"5min":0.626,"15min":0.626},"last":0.371}]}]},{"osd":3,"up_from":14,"seq":60129542491,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":104396,"kb_used_data":37124,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94267444,"statfs":{"total":96636764160,"available":96529862656,"internally_reserved":0,"allocated":38014976,"data_stored":37846259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.093,"15min":1.0820000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":3.5150000000000001,"5min":3.9849999999999999,"15min":6.1879999999999997},"last":1.377},{"interface":"front","average":{"1min":1.302,"5min":1.2929999999999999,"15min":1.278},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":4.7430000000000003,"5min":8.6639999999999997,"15min":8.6639999999999997},"last":1.107}]},{"osd":1,"last update":"Sun Jan 21 22:52:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.3109999999999999,"15min":1.26},"min":{"1min":0.77500000000000002,"5min":0.751,"15min":0.67000000000000004},"max":{"1min":3.8799999999999999,"5min":17.492999999999999,"15min":17.492999999999999},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.5609999999999999,"5min":1.472,"15min":1.4139999999999999},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":5.7270000000000003,"5min":8.9629999999999992,"15min":8.9629999999999992},"last":1.274}]},{"osd":2,"last update":"Sun Jan 21 22:52:41 2024","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.4330000000000001,"15min":1.377},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":6.2119999999999997,"5min":8.5530000000000008,"15min":8.5530000000000008},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.484,"5min":1.581,"15min":1.5209999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":4.3049999999999997,"5min":17.664999999999999,"15min":17.664999999999999},"last":1.415}]},{"osd":4,"last update":"Sun Jan 21 22:52:41 2024","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.1879999999999999,"15min":1.1479999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":4.226,"5min":4.5099999999999998,"15min":4.5099999999999998},"last":1.157},{"interface":"front","average":{"1min":1.25,"5min":1.2,"15min":1.1539999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":6.0110000000000001,"5min":6.0110000000000001,"15min":6.0110000000000001},"last":1.319}]},{"osd":5,"last update":"Sun Jan 21 22:52:41 2024","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.216,"15min":1.2010000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":9.2370000000000001},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.3799999999999999,"15min":1.3029999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":4.343,"5min":8.7420000000000009,"15min":8.7420000000000009},"last":1.9039999999999999}]},{"osd":6,"last update":"Sun Jan 21 22:52:41 2024","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.373,"15min":1.3660000000000001},"min":{"1min":0.89800000000000002,"5min":0.874,"15min":0.85899999999999999},"max":{"1min":4.6230000000000002,"5min":4.6230000000000002,"15min":9.3719999999999999},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.4390000000000001,"5min":1.423,"15min":1.3799999999999999},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":6.1109999999999998,"5min":6.1109999999999998,"15min":6.1109999999999998},"last":1.9730000000000001}]},{"osd":7,"last update":"Sun Jan 21 22:52:41 2024","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.5069999999999999,"15min":1.498},"min":{"1min":1.1339999999999999,"5min":0.97199999999999998,"15min":0.86099999999999999},"max":{"1min":4.5359999999999996,"5min":4.5359999999999996,"15min":9.4610000000000003},"last":1.698},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.5900000000000001,"15min":1.5169999999999999},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":5.242,"5min":8.8719999999999999,"15min":8.8719999999999999},"last":1.8360000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542489,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61776,"kb_used_data":12552,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94310064,"statfs":{"total":96636764160,"available":96573505536,"internally_reserved":0,"allocated":12853248,"data_stored":12680525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0509999999999999,"15min":1.04},"min":{"1min":0.79100000000000004,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.5369999999999999,"5min":1.9410000000000001,"15min":2.0419999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":1.349,"5min":1.306,"15min":1.2769999999999999},"min":{"1min":0.95599999999999996,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.6299999999999999,"5min":2.2770000000000001,"15min":2.2770000000000001},"last":0.70399999999999996}]},{"osd":1,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.29,"15min":1.2629999999999999},"min":{"1min":0.88400000000000001,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.605,"5min":1.9399999999999999,"15min":2.234},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.3540000000000001,"15min":1.3759999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.8540000000000001,"5min":2.411,"15min":2.411},"last":1.0469999999999999}]},{"osd":3,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.27,"15min":1.296},"min":{"1min":0.96799999999999997,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.556,"5min":1.788,"15min":1.9690000000000001},"last":0.624},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.429,"15min":1.4410000000000001},"min":{"1min":1.2230000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.681,"5min":2.0659999999999998,"15min":2.1320000000000001},"last":0.78000000000000003}]},{"osd":4,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.149,"15min":1.155},"min":{"1min":0.90100000000000002,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":1.345,"5min":1.776,"15min":2.0259999999999998},"last":0.876},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.0720000000000001,"15min":1.0820000000000001},"min":{"1min":0.69999999999999996,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.4199999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.83299999999999996}]},{"osd":5,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0469999999999999,"15min":1.0620000000000001},"min":{"1min":0.79900000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.415,"5min":1.623,"15min":1.669},"last":0.753},{"interface":"front","average":{"1min":1.244,"5min":1.2809999999999999,"15min":1.29},"min":{"1min":1.0860000000000001,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5720000000000001,"5min":1.8859999999999999,"15min":2.125},"last":0.91900000000000004}]},{"osd":6,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.244,"15min":1.25},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.6160000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.2130000000000001,"15min":1.254},"min":{"1min":0.96799999999999997,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.5880000000000001,"5min":1.8600000000000001,"15min":1.8919999999999999},"last":0.81100000000000005}]},{"osd":7,"last update":"Sun Jan 21 22:52:28 2024","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.377,"15min":1.4019999999999999},"min":{"1min":1.034,"5min":0.748,"15min":0.748},"max":{"1min":1.7450000000000001,"5min":2.6160000000000001,"15min":2.6160000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3640000000000001,"15min":1.391},"min":{"1min":1.077,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.694,"5min":1.9139999999999999,"15min":2.004},"last":0.85399999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542490,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76576,"kb_used_data":23192,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94295264,"statfs":{"total":96636764160,"available":96558350336,"internally_reserved":0,"allocated":23748608,"data_stored":23575404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jan 21 22:52:48 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":1.05,"15min":1.044},"min":{"1min":0.70299999999999996,"5min":0.62,"15min":0.62},"max":{"1min":1.4450000000000001,"5min":2.3839999999999999,"15min":3.613},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.2290000000000001,"15min":1.2210000000000001},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.63800000000000001},"max":{"1min":1.4890000000000001,"5min":3.3879999999999999,"15min":3.3879999999999999},"last":1.391}]},{"osd":2,"last update":"Sun Jan 21 22:52:48 2024","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.206,"15min":1.2350000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.6459999999999999,"5min":3.056,"15min":4.2309999999999999},"last":1.425},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3480000000000001,"15min":1.3540000000000001},"min":{"1min":0.85899999999999999,"5min":0.82499999999999996,"15min":0.76700000000000002},"max":{"1min":1.6579999999999999,"5min":2.8159999999999998,"15min":3.2450000000000001},"last":1.1259999999999999}]},{"osd":3,"last update":"Sun Jan 21 22:52:48 2024","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.421,"15min":1.4219999999999999},"min":{"1min":0.83999999999999997,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.605,"5min":4.0890000000000004,"15min":4.0890000000000004},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3400000000000001,"15min":1.355},"min":{"1min":1.0369999999999999,"5min":0.84599999999999997,"15min":0.80400000000000005},"max":{"1min":1.5649999999999999,"5min":3.8519999999999999,"15min":3.8519999999999999},"last":1.024}]},{"osd":4,"last update":"Sun Jan 21 22:52:48 2024","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.0629999999999999,"15min":1.038},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.4650000000000001,"5min":2.8639999999999999,"15min":2.8639999999999999},"last":1.214},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.2030000000000001,"15min":1.1919999999999999},"min":{"1min":0.88200000000000001,"5min":0.78600000000000003,"15min":0.77300000000000002},"max":{"1min":1.526,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":0.90100000000000002}]},{"osd":5,"last update":"Sun Jan 21 22:52:48 2024","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.0980000000000001,"15min":1.1080000000000001},"min":{"1min":0.79400000000000004,"5min":0.71199999999999997,"15min":0.71099999999999997},"max":{"1min":1.6970000000000001,"5min":3.0139999999999998,"15min":3.0139999999999998},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.258,"15min":1.252},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.823,"5min":2.972,"15min":2.972},"last":1.369}]},{"osd":6,"last update":"Sun Jan 21 22:52:48 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.282,"15min":1.294},"min":{"1min":0.92300000000000004,"5min":0.86799999999999999,"15min":0.79300000000000004},"max":{"1min":1.7969999999999999,"5min":3.121,"15min":3.121},"last":1.268},{"interface":"front","average":{"1min":1.401,"5min":1.417,"15min":1.4059999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.927,"5min":3.3410000000000002,"15min":3.3410000000000002},"last":1.2949999999999999}]},{"osd":7,"last update":"Sun Jan 21 22:52:48 2024","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.415,"15min":1.4259999999999999},"min":{"1min":1.002,"5min":0.89100000000000001,"15min":0.86799999999999999},"max":{"1min":1.677,"5min":2.718,"15min":2.718},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.421,"15min":1.462},"min":{"1min":1.0289999999999999,"5min":1.002,"15min":0.93200000000000005},"max":{"1min":1.8560000000000001,"5min":3.218,"15min":3.218},"last":1.3320000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542491,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":93984,"kb_used_data":26712,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94277856,"statfs":{"total":96636764160,"available":96540524544,"internally_reserved":0,"allocated":27353088,"data_stored":27179326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0309999999999999,"15min":1.0640000000000001},"min":{"1min":0.70499999999999996,"5min":0.624,"15min":0.621},"max":{"1min":1.6080000000000001,"5min":1.6870000000000001,"15min":1.988},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.288,"5min":1.232,"15min":1.2889999999999999},"min":{"1min":0.90100000000000002,"5min":0.70199999999999996,"15min":0.67200000000000004},"max":{"1min":1.609,"5min":1.718,"15min":11.01},"last":1.2450000000000001}]},{"osd":2,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.306,"15min":1.3300000000000001},"min":{"1min":1.2050000000000001,"5min":0.83799999999999997,"15min":0.81699999999999995},"max":{"1min":1.8580000000000001,"5min":1.9219999999999999,"15min":1.9219999999999999},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.377,"5min":1.3480000000000001,"15min":1.3879999999999999},"min":{"1min":0.96199999999999997,"5min":0.84499999999999997,"15min":0.84399999999999997},"max":{"1min":1.742,"5min":1.9690000000000001,"15min":11.308999999999999},"last":1.647}]},{"osd":3,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.268,"15min":1.3129999999999999},"min":{"1min":0.98999999999999999,"5min":0.85399999999999998,"15min":0.67000000000000004},"max":{"1min":1.573,"5min":1.671,"15min":9.2929999999999993},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.462,"5min":1.4390000000000001,"15min":1.4850000000000001},"min":{"1min":1.1679999999999999,"5min":1.01,"15min":0.91700000000000004},"max":{"1min":1.696,"5min":2.0390000000000001,"15min":11.4},"last":1.3400000000000001}]},{"osd":4,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0660000000000001,"15min":1.0860000000000001},"min":{"1min":0.82099999999999995,"5min":0.70999999999999996,"15min":0.66200000000000003},"max":{"1min":1.3200000000000001,"5min":1.593,"15min":2.0110000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.157,"15min":1.2050000000000001},"min":{"1min":0.94099999999999995,"5min":0.83899999999999997,"15min":0.77300000000000002},"max":{"1min":1.823,"5min":1.823,"15min":11.345000000000001},"last":1.1319999999999999}]},{"osd":5,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":1.0289999999999999,"15min":1.0549999999999999},"min":{"1min":0.76200000000000001,"5min":0.71699999999999997,"15min":0.70299999999999996},"max":{"1min":1.498,"5min":1.7470000000000001,"15min":2.0030000000000001},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2190000000000001,"15min":1.266},"min":{"1min":0.84399999999999997,"5min":0.82799999999999996,"15min":0.66300000000000003},"max":{"1min":1.4099999999999999,"5min":1.6539999999999999,"15min":2.1509999999999998},"last":1.581}]},{"osd":6,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1819999999999999,"15min":1.2130000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.369,"5min":1.871,"15min":2.198},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.353,"5min":1.3540000000000001,"15min":1.397},"min":{"1min":1.0269999999999999,"5min":0.91000000000000003,"15min":0.872},"max":{"1min":1.74,"5min":1.8120000000000001,"15min":2.2509999999999999},"last":1.71}]},{"osd":7,"last update":"Sun Jan 21 22:52:31 2024","interfaces":[{"interface":"back","average":{"1min":1.48,"5min":1.3680000000000001,"15min":1.4219999999999999},"min":{"1min":1.1839999999999999,"5min":0.96299999999999997,"15min":0.85799999999999998},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":11.374000000000001},"last":1.758},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3460000000000001,"15min":1.369},"min":{"1min":0.97899999999999998,"5min":0.94999999999999996,"15min":0.875},"max":{"1min":1.855,"5min":1.9810000000000001,"15min":2.3319999999999999},"last":1.48}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":27090944,"data_stored":27082385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22892544,"data_stored":22888095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12591104,"data_stored":12583584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":37752832,"data_stored":37749318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12599296,"data_stored":12584256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":25788416,"data_stored":25776663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":13205504,"data_stored":13193783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-01-21T22:53:16.218 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-01-21T22:53:16.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.451+0000 7f6eb9200640 1 Processor -- start 2024-01-21T22:53:16.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.451+0000 7f6eb9200640 1 -- start start 2024-01-21T22:53:16.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.452+0000 7f6eb9200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6eb4150b90 0x7f6eb40c2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:16.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.452+0000 7f6eb9200640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6eb4071380 0x7f6eb40caf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:16.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.452+0000 7f6eb9200640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb40c75e0 con 0x7f6eb40719f0 2024-01-21T22:53:16.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.452+0000 7f6eb9200640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb40c6d20 con 0x7f6eb4071380 2024-01-21T22:53:16.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.452+0000 7f6eb9200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb4070dc0 con 0x7f6eb4150b90 2024-01-21T22:53:16.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.453+0000 7f6eb1e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6eb4150b90 0x7f6eb40c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:16.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.453+0000 7f6eb1400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f6eb4071380 0x7f6eb40caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:16.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.453+0000 7f6eb2800640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f6eb40719f0 0x7f6eb4159c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:34980/0 (socket says 172.21.15.138:34980) 2024-01-21T22:53:16.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.453+0000 7f6eb2800640 1 -- 172.21.15.138:0/2986383165 learned_addr learned my addr 172.21.15.138:0/2986383165 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:53:16.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.456+0000 7f6eb1e00640 1 -- 172.21.15.138:0/2986383165 >> v2:172.21.15.152:3300/0 conn(0x7f6eb4071380 msgr2=0x7f6eb40caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:53:16.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.456+0000 7f6eb1e00640 1 --2- 172.21.15.138:0/2986383165 >> v2:172.21.15.152:3300/0 conn(0x7f6eb4071380 0x7f6eb40caf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:16.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.456+0000 7f6eb1e00640 1 -- 172.21.15.138:0/2986383165 >> v1:172.21.15.138:6789/0 conn(0x7f6eb40719f0 legacy=0x7f6eb4159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:16.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.456+0000 7f6eb1e00640 1 -- 172.21.15.138:0/2986383165 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb40710a0 con 0x7f6eb4150b90 2024-01-21T22:53:16.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.458+0000 7f6eb1e00640 1 --2- 172.21.15.138:0/2986383165 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6eb4150b90 0x7f6eb40c2540 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40097a0 tx=0x7f6ea402dae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41a5bd77c9de34fa server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:16.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.461+0000 7f6eb0a00640 1 -- 172.21.15.138:0/2986383165 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea402f8a0 con 0x7f6eb4150b90 2024-01-21T22:53:16.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.462+0000 7f6eb0a00640 1 -- 172.21.15.138:0/2986383165 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea402fa00 con 0x7f6eb4150b90 2024-01-21T22:53:16.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.462+0000 7f6eb0a00640 1 -- 172.21.15.138:0/2986383165 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea402fd90 con 0x7f6eb4150b90 2024-01-21T22:53:16.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.462+0000 7f6eb9200640 1 -- 172.21.15.138:0/2986383165 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6eb4150b90 msgr2=0x7f6eb40c2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:16.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.462+0000 7f6eb9200640 1 --2- 172.21.15.138:0/2986383165 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6eb4150b90 0x7f6eb40c2540 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40097a0 tx=0x7f6ea402dae0 comp rx=0 tx=0).stop 2024-01-21T22:53:16.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.462+0000 7f6eb9200640 1 -- 172.21.15.138:0/2986383165 shutdown_connections 2024-01-21T22:53:16.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.462+0000 7f6eb9200640 1 --2- 172.21.15.138:0/2986383165 >> v2:172.21.15.152:3300/0 conn(0x7f6eb4071380 0x7f6eb40caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:16.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.462+0000 7f6eb9200640 1 --2- 172.21.15.138:0/2986383165 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6eb4150b90 0x7f6eb40c2540 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:16.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.462+0000 7f6eb9200640 1 -- 172.21.15.138:0/2986383165 >> 172.21.15.138:0/2986383165 conn(0x7f6eb4072a20 msgr2=0x7f6eb4066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:53:16.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.462+0000 7f6eb9200640 1 -- 172.21.15.138:0/2986383165 shutdown_connections 2024-01-21T22:53:16.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.463+0000 7f6eb9200640 1 -- 172.21.15.138:0/2986383165 wait complete. 2024-01-21T22:53:16.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.464+0000 7f6eb9200640 1 Processor -- start 2024-01-21T22:53:16.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.464+0000 7f6eb9200640 1 -- start start 2024-01-21T22:53:16.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.465+0000 7f6eb9200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb4071380 0x7f6eb4153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:16.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.465+0000 7f6eb9200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6eb40719f0 0x7f6eb4158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:16.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.465+0000 7f6eb9200640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6eb4150b90 0x7f6eb41469d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:16.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.465+0000 7f6eb9200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb4153160 con 0x7f6eb4071380 2024-01-21T22:53:16.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.465+0000 7f6eb9200640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb41593f0 con 0x7f6eb40719f0 2024-01-21T22:53:16.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.465+0000 7f6eb9200640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb41f2dd0 con 0x7f6eb4150b90 2024-01-21T22:53:16.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.466+0000 7f6eb2800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6eb4150b90 0x7f6eb41469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:16.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.466+0000 7f6eb1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb4071380 0x7f6eb4153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:16.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.466+0000 7f6eb1400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6eb40719f0 0x7f6eb4158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:16.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.466+0000 7f6eb1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb4071380 0x7f6eb4153cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39384/0 (socket says 172.21.15.138:39384) 2024-01-21T22:53:16.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.466+0000 7f6eb1e00640 1 -- 172.21.15.138:0/544533665 learned_addr learned my addr 172.21.15.138:0/544533665 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:53:16.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.470+0000 7f6eb2800640 1 -- 172.21.15.138:0/544533665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6eb40719f0 msgr2=0x7f6eb4158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:16.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.470+0000 7f6eb2800640 1 --2- 172.21.15.138:0/544533665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6eb40719f0 0x7f6eb4158940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:16.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.470+0000 7f6eb2800640 1 -- 172.21.15.138:0/544533665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb4071380 msgr2=0x7f6eb4153cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:16.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.470+0000 7f6eb2800640 1 --2- 172.21.15.138:0/544533665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb4071380 0x7f6eb4153cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:16.471 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.470+0000 7f6eb2800640 1 -- 172.21.15.138:0/544533665 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea4009400 con 0x7f6eb4150b90 2024-01-21T22:53:16.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.472+0000 7f6eb2800640 1 --2- 172.21.15.138:0/544533665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6eb4150b90 0x7f6eb41469d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00caa0 tx=0x7f6eac00cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:16.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.475+0000 7f6eaa000640 1 -- 172.21.15.138:0/544533665 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac00ac00 con 0x7f6eb4150b90 2024-01-21T22:53:16.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.475+0000 7f6eb9200640 1 -- 172.21.15.138:0/544533665 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb41f3070 con 0x7f6eb4150b90 2024-01-21T22:53:16.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.475+0000 7f6eb9200640 1 -- 172.21.15.138:0/544533665 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb41f3630 con 0x7f6eb4150b90 2024-01-21T22:53:16.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.476+0000 7f6eb9200640 1 -- 172.21.15.138:0/544533665 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb404eb10 con 0x7f6eb4150b90 2024-01-21T22:53:16.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.477+0000 7f6eaa000640 1 -- 172.21.15.138:0/544533665 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac001b20 con 0x7f6eb4150b90 2024-01-21T22:53:16.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.477+0000 7f6eaa000640 1 -- 172.21.15.138:0/544533665 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac01c440 con 0x7f6eb4150b90 2024-01-21T22:53:16.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.497+0000 7f6eaa000640 1 -- 172.21.15.138:0/544533665 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f6eac012400 con 0x7f6eb4150b90 2024-01-21T22:53:16.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.498+0000 7f6eaa000640 1 --2- 172.21.15.138:0/544533665 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6e8c06b330 0x7f6e8c06d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:16.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.498+0000 7f6eaa000640 1 -- 172.21.15.138:0/544533665 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f6eac058c40 con 0x7f6eb4150b90 2024-01-21T22:53:16.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.499+0000 7f6eb1e00640 1 --2- 172.21.15.138:0/544533665 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6e8c06b330 0x7f6e8c06d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:16.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.500+0000 7f6eb1e00640 1 --2- 172.21.15.138:0/544533665 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6e8c06b330 0x7f6e8c06d7f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6ea402db20 tx=0x7f6ea4008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:16.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.738+0000 7f6eaa000640 1 -- 172.21.15.138:0/544533665 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f6eac054f80 con 0x7f6eb4150b90 2024-01-21T22:53:16.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:16.940+0000 7f6eb9200640 1 -- 172.21.15.138:0/544533665 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f6eb40717d0 con 0x7f6eb4150b90 2024-01-21T22:53:17.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.268+0000 7f6eaa000640 1 -- 172.21.15.138:0/544533665 <== mon.2 v2:172.21.15.152:3301/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6eac05cce0 con 0x7f6eb4150b90 2024-01-21T22:53:17.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.268+0000 7f6eaa000640 1 -- 172.21.15.138:0/544533665 <== mon.2 v2:172.21.15.152:3301/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) v1 ==== 123+0+0 (secure 0 0 0) 0x7f6eac0585a0 con 0x7f6eb4150b90 2024-01-21T22:53:17.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.272+0000 7f6eb9200640 1 -- 172.21.15.138:0/544533665 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6e8c06b330 msgr2=0x7f6e8c06d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:17.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.272+0000 7f6eb9200640 1 --2- 172.21.15.138:0/544533665 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6e8c06b330 0x7f6e8c06d7f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6ea402db20 tx=0x7f6ea4008030 comp rx=0 tx=0).stop 2024-01-21T22:53:17.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.272+0000 7f6eb9200640 1 -- 172.21.15.138:0/544533665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6eb4150b90 msgr2=0x7f6eb41469d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:17.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.272+0000 7f6eb9200640 1 --2- 172.21.15.138:0/544533665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6eb4150b90 0x7f6eb41469d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00caa0 tx=0x7f6eac00cf60 comp rx=0 tx=0).stop 2024-01-21T22:53:17.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.272+0000 7f6eb9200640 1 -- 172.21.15.138:0/544533665 shutdown_connections 2024-01-21T22:53:17.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.272+0000 7f6eb9200640 1 --2- 172.21.15.138:0/544533665 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f6e8c06b330 0x7f6e8c06d7f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:17.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.272+0000 7f6eb9200640 1 --2- 172.21.15.138:0/544533665 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f6eb4150b90 0x7f6eb41469d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:17.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.272+0000 7f6eb9200640 1 --2- 172.21.15.138:0/544533665 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f6eb40719f0 0x7f6eb4158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:17.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.272+0000 7f6eb9200640 1 --2- 172.21.15.138:0/544533665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb4071380 0x7f6eb4153cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:17.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.272+0000 7f6eb9200640 1 -- 172.21.15.138:0/544533665 >> 172.21.15.138:0/544533665 conn(0x7f6eb4072a20 msgr2=0x7f6eb406a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:53:17.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.272+0000 7f6eb9200640 1 -- 172.21.15.138:0/544533665 shutdown_connections 2024-01-21T22:53:17.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.272+0000 7f6eb9200640 1 -- 172.21.15.138:0/544533665 wait complete. 2024-01-21T22:53:17.302 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-01-21T22:53:17.302 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-21T22:53:17.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.494+0000 7f7e75c00640 1 Processor -- start 2024-01-21T22:53:17.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.495+0000 7f7e75c00640 1 -- start start 2024-01-21T22:53:17.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.496+0000 7f7e75c00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e7006ad40 0x7f7e7006b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:17.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.496+0000 7f7e75c00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e70157740 con 0x7f7e70150a00 2024-01-21T22:53:17.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.496+0000 7f7e75c00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7006b6b0 con 0x7f7e700716e0 2024-01-21T22:53:17.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.496+0000 7f7e75c00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e700c4cc0 con 0x7f7e7006ad40 2024-01-21T22:53:17.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.497+0000 7f7e6ea00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f7e70150a00 0x7f7e700c25b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:34994/0 (socket says 172.21.15.138:34994) 2024-01-21T22:53:17.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.497+0000 7f7e6ea00640 1 -- 172.21.15.138:0/1296653182 learned_addr learned my addr 172.21.15.138:0/1296653182 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:53:17.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.497+0000 7f7e6f400640 1 --2- 172.21.15.138:0/1296653182 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e7006ad40 0x7f7e7006b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:17.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.499+0000 7f7e6d600640 1 -- 172.21.15.138:0/1296653182 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4073505049 0 0) 0x7f7e7006b6b0 con 0x7f7e700716e0 2024-01-21T22:53:17.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.499+0000 7f7e6d600640 1 -- 172.21.15.138:0/1296653182 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e50003520 con 0x7f7e700716e0 2024-01-21T22:53:17.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.500+0000 7f7e6f400640 1 -- 172.21.15.138:0/1296653182 >> v1:172.21.15.152:6789/0 conn(0x7f7e700716e0 legacy=0x7f7e700c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:17.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.500+0000 7f7e6f400640 1 -- 172.21.15.138:0/1296653182 >> v1:172.21.15.138:6789/0 conn(0x7f7e70150a00 legacy=0x7f7e700c25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:17.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.500+0000 7f7e6f400640 1 -- 172.21.15.138:0/1296653182 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e70156bc0 con 0x7f7e7006ad40 2024-01-21T22:53:17.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.502+0000 7f7e6f400640 1 --2- 172.21.15.138:0/1296653182 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e7006ad40 0x7f7e7006b170 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009dc0 tx=0x7f7e6802be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=414200e12388ad93 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:17.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.506+0000 7f7e6d600640 1 -- 172.21.15.138:0/1296653182 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6802cc40 con 0x7f7e7006ad40 2024-01-21T22:53:17.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.506+0000 7f7e6d600640 1 -- 172.21.15.138:0/1296653182 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7e6802cda0 con 0x7f7e7006ad40 2024-01-21T22:53:17.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.506+0000 7f7e6d600640 1 -- 172.21.15.138:0/1296653182 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68034680 con 0x7f7e7006ad40 2024-01-21T22:53:17.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.506+0000 7f7e75c00640 1 -- 172.21.15.138:0/1296653182 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e7006ad40 msgr2=0x7f7e7006b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:17.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.506+0000 7f7e75c00640 1 --2- 172.21.15.138:0/1296653182 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e7006ad40 0x7f7e7006b170 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009dc0 tx=0x7f7e6802be90 comp rx=0 tx=0).stop 2024-01-21T22:53:17.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.506+0000 7f7e75c00640 1 -- 172.21.15.138:0/1296653182 shutdown_connections 2024-01-21T22:53:17.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.506+0000 7f7e75c00640 1 --2- 172.21.15.138:0/1296653182 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e7006ad40 0x7f7e7006b170 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:17.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.506+0000 7f7e75c00640 1 -- 172.21.15.138:0/1296653182 >> 172.21.15.138:0/1296653182 conn(0x7f7e700cb0b0 msgr2=0x7f7e700cb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:53:17.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.507+0000 7f7e75c00640 1 -- 172.21.15.138:0/1296653182 shutdown_connections 2024-01-21T22:53:17.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.507+0000 7f7e75c00640 1 -- 172.21.15.138:0/1296653182 wait complete. 2024-01-21T22:53:17.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.508+0000 7f7e75c00640 1 Processor -- start 2024-01-21T22:53:17.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.508+0000 7f7e75c00640 1 -- start start 2024-01-21T22:53:17.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.509+0000 7f7e75c00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e7006ad40 0x7f7e7006c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:17.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.509+0000 7f7e75c00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e700716e0 0x7f7e701ef750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:17.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.509+0000 7f7e75c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e70150a00 0x7f7e700c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:17.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.509+0000 7f7e75c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7006a920 con 0x7f7e70150a00 2024-01-21T22:53:17.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.509+0000 7f7e75c00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e70153ed0 con 0x7f7e7006ad40 2024-01-21T22:53:17.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.509+0000 7f7e75c00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e701ef2c0 con 0x7f7e700716e0 2024-01-21T22:53:17.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.510+0000 7f7e6ea00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e7006ad40 0x7f7e7006c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:17.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.510+0000 7f7e6f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e70150a00 0x7f7e700c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:17.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.510+0000 7f7e6e000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e700716e0 0x7f7e701ef750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:17.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.510+0000 7f7e6f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e70150a00 0x7f7e700c92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39386/0 (socket says 172.21.15.138:39386) 2024-01-21T22:53:17.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.510+0000 7f7e6f400640 1 -- 172.21.15.138:0/104510558 learned_addr learned my addr 172.21.15.138:0/104510558 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:53:17.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.514+0000 7f7e6e000640 1 -- 172.21.15.138:0/104510558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e7006ad40 msgr2=0x7f7e7006c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:17.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.514+0000 7f7e6e000640 1 --2- 172.21.15.138:0/104510558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e7006ad40 0x7f7e7006c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:17.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.514+0000 7f7e6e000640 1 -- 172.21.15.138:0/104510558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e70150a00 msgr2=0x7f7e700c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:17.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.514+0000 7f7e6e000640 1 --2- 172.21.15.138:0/104510558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e70150a00 0x7f7e700c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:17.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.514+0000 7f7e6e000640 1 -- 172.21.15.138:0/104510558 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e68009a70 con 0x7f7e700716e0 2024-01-21T22:53:17.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.517+0000 7f7e6e000640 1 --2- 172.21.15.138:0/104510558 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e700716e0 0x7f7e701ef750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800c3c0 tx=0x7f7e5800c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:17.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.520+0000 7f7e66a00640 1 -- 172.21.15.138:0/104510558 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e580052d0 con 0x7f7e700716e0 2024-01-21T22:53:17.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.521+0000 7f7e66a00640 1 -- 172.21.15.138:0/104510558 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7e58005430 con 0x7f7e700716e0 2024-01-21T22:53:17.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.521+0000 7f7e66a00640 1 -- 172.21.15.138:0/104510558 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58003850 con 0x7f7e700716e0 2024-01-21T22:53:17.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.521+0000 7f7e75c00640 1 -- 172.21.15.138:0/104510558 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e700c8eb0 con 0x7f7e700716e0 2024-01-21T22:53:17.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.521+0000 7f7e75c00640 1 -- 172.21.15.138:0/104510558 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e700c14a0 con 0x7f7e700716e0 2024-01-21T22:53:17.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.522+0000 7f7e75c00640 1 -- 172.21.15.138:0/104510558 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e701ef430 con 0x7f7e700716e0 2024-01-21T22:53:17.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.543+0000 7f7e66a00640 1 -- 172.21.15.138:0/104510558 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f7e580039b0 con 0x7f7e700716e0 2024-01-21T22:53:17.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.544+0000 7f7e66a00640 1 --2- 172.21.15.138:0/104510558 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7e4806b380 0x7f7e4806d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:17.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.545+0000 7f7e6ea00640 1 --2- 172.21.15.138:0/104510558 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7e4806b380 0x7f7e4806d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:17.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.545+0000 7f7e6ea00640 1 --2- 172.21.15.138:0/104510558 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7e4806b380 0x7f7e4806d840 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7e60005880 tx=0x7f7e60009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:17.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.770+0000 7f7e66a00640 1 -- 172.21.15.138:0/104510558 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7e58057d80 con 0x7f7e700716e0 2024-01-21T22:53:17.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:17.781+0000 7f7e66a00640 1 -- 172.21.15.138:0/104510558 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f7e58054030 con 0x7f7e700716e0 2024-01-21T22:53:18.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.039+0000 7f7e75c00640 1 -- 172.21.15.138:0/104510558 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e7004eaa0 con 0x7f7e700716e0 2024-01-21T22:53:18.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.051+0000 7f7e66a00640 1 -- 172.21.15.138:0/104510558 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+6530 (secure 0 0 0) 0x7f7e58017020 con 0x7f7e700716e0 2024-01-21T22:53:18.053 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:53:18.053 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/3112363661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":3112363661},{"type":"v1","addr":"172.21.15.152:6837","nonce":3112363661}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4223,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/3222259940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":3222259940},{"type":"v1","addr":"172.21.15.152:6839","nonce":3222259940}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/909784319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":909784319},{"type":"v1","addr":"172.21.15.138:6833","nonce":909784319}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4229,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3318550662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3318550662},{"type":"v1","addr":"172.21.15.138:6835","nonce":3318550662}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2986531362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2986531362},{"type":"v1","addr":"172.21.15.138:6837","nonce":2986531362}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-21T22:25:04.967250+0000","modified":"2024-01-21T22:25:12.598970+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":5,"state":"up:active","state_seq":5,"addr":"172.21.15.152:6835/2663179600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2663179600},{"type":"v1","addr":"172.21.15.152:6835","nonce":2663179600}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-21T22:53:18.053 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 10 2024-01-21T22:53:18.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.054+0000 7f7e75c00640 1 -- 172.21.15.138:0/104510558 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7e4806b380 msgr2=0x7f7e4806d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:18.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.054+0000 7f7e75c00640 1 --2- 172.21.15.138:0/104510558 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7e4806b380 0x7f7e4806d840 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7e60005880 tx=0x7f7e60009040 comp rx=0 tx=0).stop 2024-01-21T22:53:18.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.054+0000 7f7e75c00640 1 -- 172.21.15.138:0/104510558 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e700716e0 msgr2=0x7f7e701ef750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:18.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.054+0000 7f7e75c00640 1 --2- 172.21.15.138:0/104510558 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e700716e0 0x7f7e701ef750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800c3c0 tx=0x7f7e5800c880 comp rx=0 tx=0).stop 2024-01-21T22:53:18.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.055+0000 7f7e75c00640 1 -- 172.21.15.138:0/104510558 shutdown_connections 2024-01-21T22:53:18.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.055+0000 7f7e75c00640 1 --2- 172.21.15.138:0/104510558 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7e4806b380 0x7f7e4806d840 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:18.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.055+0000 7f7e75c00640 1 --2- 172.21.15.138:0/104510558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e70150a00 0x7f7e700c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:18.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.055+0000 7f7e75c00640 1 --2- 172.21.15.138:0/104510558 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7e700716e0 0x7f7e701ef750 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:18.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.055+0000 7f7e75c00640 1 --2- 172.21.15.138:0/104510558 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7e7006ad40 0x7f7e7006c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:18.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.055+0000 7f7e75c00640 1 -- 172.21.15.138:0/104510558 >> 172.21.15.138:0/104510558 conn(0x7f7e700cb0b0 msgr2=0x7f7e700c83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:53:18.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.055+0000 7f7e75c00640 1 -- 172.21.15.138:0/104510558 shutdown_connections 2024-01-21T22:53:18.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.055+0000 7f7e75c00640 1 -- 172.21.15.138:0/104510558 wait complete. 2024-01-21T22:53:18.077 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-01-21T22:53:18.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.281+0000 7f4966a00640 1 Processor -- start 2024-01-21T22:53:18.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.281+0000 7f4966a00640 1 -- start start 2024-01-21T22:53:18.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.283+0000 7f4966a00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f496006b8d0 0x7f49600c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:18.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.283+0000 7f4966a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49600ca000 con 0x7f496006c0a0 2024-01-21T22:53:18.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.283+0000 7f4966a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f496006ad30 con 0x7f4960153a50 2024-01-21T22:53:18.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.283+0000 7f4966a00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49600cb8d0 con 0x7f496006b8d0 2024-01-21T22:53:18.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.284+0000 7f4965600640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f4960153a50 0x7f49600cbb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:45006/0 (socket says 172.21.15.138:45006) 2024-01-21T22:53:18.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.284+0000 7f4965600640 1 -- 172.21.15.138:0/1295164509 learned_addr learned my addr 172.21.15.138:0/1295164509 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:53:18.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.286+0000 7f495fe00640 1 -- 172.21.15.138:0/1295164509 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3144338620 0 0) 0x7f496006ad30 con 0x7f4960153a50 2024-01-21T22:53:18.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.286+0000 7f495fe00640 1 -- 172.21.15.138:0/1295164509 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4940003520 con 0x7f4960153a50 2024-01-21T22:53:18.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.290+0000 7f495fe00640 1 -- 172.21.15.138:0/1295164509 <== mon.1 v1:172.21.15.152:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1845074844 0 0) 0x7f4940003520 con 0x7f4960153a50 2024-01-21T22:53:18.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.290+0000 7f495fe00640 1 -- 172.21.15.138:0/1295164509 >> v2:172.21.15.138:3300/0 conn(0x7f496006b8d0 msgr2=0x7f49600c9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:53:18.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.290+0000 7f495fe00640 1 --2- 172.21.15.138:0/1295164509 >> v2:172.21.15.138:3300/0 conn(0x7f496006b8d0 0x7f49600c9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:18.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.290+0000 7f495fe00640 1 -- 172.21.15.138:0/1295164509 >> v1:172.21.15.138:6789/0 conn(0x7f496006c0a0 legacy=0x7f496006b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:53:18.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.290+0000 7f495fe00640 1 -- 172.21.15.138:0/1295164509 --> v1:172.21.15.152:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496015d120 con 0x7f4960153a50 2024-01-21T22:53:18.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.290+0000 7f495fe00640 1 -- 172.21.15.138:0/1295164509 <== mon.1 v1:172.21.15.152:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1288616872 0 0) 0x7f4950001670 con 0x7f4960153a50 2024-01-21T22:53:18.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.290+0000 7f495fe00640 1 -- 172.21.15.138:0/1295164509 >> v1:172.21.15.152:6789/0 conn(0x7f4960153a50 legacy=0x7f49600cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:18.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.290+0000 7f495fe00640 1 --2- 172.21.15.138:0/1295164509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49400039c0 0x7f4940003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:18.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.290+0000 7f495fe00640 1 --2- 172.21.15.138:0/1295164509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49400047c0 0x7f4940004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:18.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.290+0000 7f495fe00640 1 --2- 172.21.15.138:0/1295164509 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4940005330 0x7f494000b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:18.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.290+0000 7f495fe00640 1 -- 172.21.15.138:0/1295164509 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49600cb8d0 con 0x7f49400047c0 2024-01-21T22:53:18.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.290+0000 7f495fe00640 1 -- 172.21.15.138:0/1295164509 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494000c280 con 0x7f49400039c0 2024-01-21T22:53:18.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.290+0000 7f495fe00640 1 -- 172.21.15.138:0/1295164509 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494000c440 con 0x7f4940005330 2024-01-21T22:53:18.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.291+0000 7f4966000640 1 --2- 172.21.15.138:0/1295164509 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4940005330 0x7f494000b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:18.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.293+0000 7f4965600640 1 --2- 172.21.15.138:0/1295164509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49400039c0 0x7f4940003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:18.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.294+0000 7f4966000640 1 -- 172.21.15.138:0/1295164509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49400039c0 msgr2=0x7f4940003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:53:18.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.294+0000 7f4966000640 1 --2- 172.21.15.138:0/1295164509 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f49400039c0 0x7f4940003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:18.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.294+0000 7f4966000640 1 -- 172.21.15.138:0/1295164509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49400047c0 msgr2=0x7f4940004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:53:18.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.294+0000 7f4966000640 1 --2- 172.21.15.138:0/1295164509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f49400047c0 0x7f4940004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:18.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.294+0000 7f4966000640 1 -- 172.21.15.138:0/1295164509 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4940003520 con 0x7f4940005330 2024-01-21T22:53:18.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.298+0000 7f4966000640 1 -- 172.21.15.138:0/1295164509 reap_dead start 2024-01-21T22:53:18.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.299+0000 7f4966000640 1 --2- 172.21.15.138:0/1295164509 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4940005330 0x7f494000b9a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4958009e40 tx=0x7f49400057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d08fc17bd9aef70 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:18.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.303+0000 7f495fe00640 1 -- 172.21.15.138:0/1295164509 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495802cd90 con 0x7f4940005330 2024-01-21T22:53:18.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.303+0000 7f495fe00640 1 -- 172.21.15.138:0/1295164509 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4958003970 con 0x7f4940005330 2024-01-21T22:53:18.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.303+0000 7f495fe00640 1 -- 172.21.15.138:0/1295164509 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4958003d40 con 0x7f4940005330 2024-01-21T22:53:18.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.304+0000 7f4966a00640 1 -- 172.21.15.138:0/1295164509 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4940005330 msgr2=0x7f494000b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:18.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.304+0000 7f4966a00640 1 --2- 172.21.15.138:0/1295164509 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4940005330 0x7f494000b9a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4958009e40 tx=0x7f49400057f0 comp rx=0 tx=0).stop 2024-01-21T22:53:18.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.304+0000 7f4966a00640 1 -- 172.21.15.138:0/1295164509 shutdown_connections 2024-01-21T22:53:18.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.304+0000 7f4966a00640 1 --2- 172.21.15.138:0/1295164509 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f4940005330 0x7f494000b9a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:18.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.304+0000 7f4966a00640 1 -- 172.21.15.138:0/1295164509 >> 172.21.15.138:0/1295164509 conn(0x7f496006d240 msgr2=0x7f4960066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:53:18.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.304+0000 7f4966a00640 1 -- 172.21.15.138:0/1295164509 shutdown_connections 2024-01-21T22:53:18.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.304+0000 7f4966a00640 1 -- 172.21.15.138:0/1295164509 wait complete. 2024-01-21T22:53:18.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.306+0000 7f4966a00640 1 Processor -- start 2024-01-21T22:53:18.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.306+0000 7f4966a00640 1 -- start start 2024-01-21T22:53:18.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.307+0000 7f4966a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4940005330 0x7f496014d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:18.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.307+0000 7f4966a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f49601eefd0 0x7f496014fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:18.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.307+0000 7f4966a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4960159900 0x7f4960159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:18.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.307+0000 7f4966a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4960159470 con 0x7f4940005330 2024-01-21T22:53:18.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.307+0000 7f4966a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49601ef460 con 0x7f4960159900 2024-01-21T22:53:18.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.307+0000 7f4966a00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49601ef740 con 0x7f49601eefd0 2024-01-21T22:53:18.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.308+0000 7f4966000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4960159900 0x7f4960159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:18.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.309+0000 7f4964c00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f49601eefd0 0x7f496014fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:18.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.309+0000 7f4966000640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4960159900 0x7f4960159d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:44660/0 (socket says 172.21.15.138:44660) 2024-01-21T22:53:18.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.309+0000 7f4966000640 1 -- 172.21.15.138:0/1159732449 learned_addr learned my addr 172.21.15.138:0/1159732449 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:53:18.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.312+0000 7f4966000640 1 -- 172.21.15.138:0/1159732449 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f49601eefd0 msgr2=0x7f496014fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:18.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.312+0000 7f4966000640 1 --2- 172.21.15.138:0/1159732449 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f49601eefd0 0x7f496014fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:18.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.312+0000 7f4966000640 1 -- 172.21.15.138:0/1159732449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4940005330 msgr2=0x7f496014d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:18.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.312+0000 7f4966000640 1 --2- 172.21.15.138:0/1159732449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4940005330 0x7f496014d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:18.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.312+0000 7f4966000640 1 -- 172.21.15.138:0/1159732449 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49601efa20 con 0x7f4960159900 2024-01-21T22:53:18.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.314+0000 7f4966000640 1 --2- 172.21.15.138:0/1159732449 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4960159900 0x7f4960159d90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4958009f30 tx=0x7f4958004bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:18.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.317+0000 7f495d600640 1 -- 172.21.15.138:0/1159732449 <== mon.1 v2:172.21.15.152:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495803b040 con 0x7f4960159900 2024-01-21T22:53:18.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.317+0000 7f495d600640 1 -- 172.21.15.138:0/1159732449 <== mon.1 v2:172.21.15.152:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f4958040610 con 0x7f4960159900 2024-01-21T22:53:18.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.317+0000 7f495d600640 1 -- 172.21.15.138:0/1159732449 <== mon.1 v2:172.21.15.152:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49580409e0 con 0x7f4960159900 2024-01-21T22:53:18.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.317+0000 7f4966a00640 1 -- 172.21.15.138:0/1159732449 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49601efd00 con 0x7f4960159900 2024-01-21T22:53:18.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.318+0000 7f4966a00640 1 -- 172.21.15.138:0/1159732449 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f496014ff70 con 0x7f4960159900 2024-01-21T22:53:18.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.319+0000 7f4966a00640 1 -- 172.21.15.138:0/1159732449 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49601efef0 con 0x7f4960159900 2024-01-21T22:53:18.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.339+0000 7f495d600640 1 -- 172.21.15.138:0/1159732449 <== mon.1 v2:172.21.15.152:3300/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f495804a3d0 con 0x7f4960159900 2024-01-21T22:53:18.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.340+0000 7f495d600640 1 --2- 172.21.15.138:0/1159732449 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f493806b330 0x7f493806d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:18.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.341+0000 7f4965600640 1 --2- 172.21.15.138:0/1159732449 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f493806b330 0x7f493806d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:18.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.342+0000 7f4965600640 1 --2- 172.21.15.138:0/1159732449 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f493806b330 0x7f493806d7f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4950001050 tx=0x7f4950001170 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:18.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.572+0000 7f495d600640 1 -- 172.21.15.138:0/1159732449 <== mon.1 v2:172.21.15.152:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f495807a300 con 0x7f4960159900 2024-01-21T22:53:18.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.582+0000 7f495d600640 1 -- 172.21.15.138:0/1159732449 <== mon.1 v2:172.21.15.152:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f4958076640 con 0x7f4960159900 2024-01-21T22:53:18.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.831+0000 7f4966a00640 1 -- 172.21.15.138:0/1159732449 --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f496004ea30 con 0x7f4960159900 2024-01-21T22:53:18.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.899+0000 7f495d600640 1 -- 172.21.15.138:0/1159732449 <== mon.1 v2:172.21.15.152:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+6530 (secure 0 0 0) 0x7f4958046030 con 0x7f4960159900 2024-01-21T22:53:18.900 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:53:18.900 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":10,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4185,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6837/3112363661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6836","nonce":3112363661},{"type":"v1","addr":"172.21.15.152:6837","nonce":3112363661}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4223,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.152:6839/3222259940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6838","nonce":3222259940},{"type":"v1","addr":"172.21.15.152:6839","nonce":3222259940}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4226,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6833/909784319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":909784319},{"type":"v1","addr":"172.21.15.138:6833","nonce":909784319}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4229,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/3318550662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":3318550662},{"type":"v1","addr":"172.21.15.138:6835","nonce":3318550662}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4235,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/2986531362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":2986531362},{"type":"v1","addr":"172.21.15.138:6837","nonce":2986531362}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-01-21T22:25:04.967250+0000","modified":"2024-01-21T22:25:12.598970+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":5,"state":"up:active","state_seq":5,"addr":"172.21.15.152:6835/2663179600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6834","nonce":2663179600},{"type":"v1","addr":"172.21.15.152:6835","nonce":2663179600}]},"join_fscid":-1,"export_targets":[],"features":4540701272860131327,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-01-21T22:53:18.901 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 10 2024-01-21T22:53:18.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.901+0000 7f4966a00640 1 -- 172.21.15.138:0/1159732449 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f493806b330 msgr2=0x7f493806d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:18.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.901+0000 7f4966a00640 1 --2- 172.21.15.138:0/1159732449 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f493806b330 0x7f493806d7f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4950001050 tx=0x7f4950001170 comp rx=0 tx=0).stop 2024-01-21T22:53:18.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.901+0000 7f4966a00640 1 -- 172.21.15.138:0/1159732449 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4960159900 msgr2=0x7f4960159d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:18.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.901+0000 7f4966a00640 1 --2- 172.21.15.138:0/1159732449 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4960159900 0x7f4960159d90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4958009f30 tx=0x7f4958004bc0 comp rx=0 tx=0).stop 2024-01-21T22:53:18.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.901+0000 7f4966a00640 1 -- 172.21.15.138:0/1159732449 shutdown_connections 2024-01-21T22:53:18.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.901+0000 7f4966a00640 1 --2- 172.21.15.138:0/1159732449 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f493806b330 0x7f493806d7f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:18.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.901+0000 7f4966a00640 1 --2- 172.21.15.138:0/1159732449 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f4960159900 0x7f4960159d90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:18.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.901+0000 7f4966a00640 1 --2- 172.21.15.138:0/1159732449 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f49601eefd0 0x7f496014fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:18.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.901+0000 7f4966a00640 1 --2- 172.21.15.138:0/1159732449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4940005330 0x7f496014d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:18.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.901+0000 7f4966a00640 1 -- 172.21.15.138:0/1159732449 >> 172.21.15.138:0/1159732449 conn(0x7f496006d240 msgr2=0x7f49600c4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:53:18.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.902+0000 7f4966a00640 1 -- 172.21.15.138:0/1159732449 shutdown_connections 2024-01-21T22:53:18.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:18.902+0000 7f4966a00640 1 -- 172.21.15.138:0/1159732449 wait complete. 2024-01-21T22:53:18.923 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-01-21T22:53:19.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.122+0000 7f2648e00640 1 Processor -- start 2024-01-21T22:53:19.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.122+0000 7f2648e00640 1 -- start start 2024-01-21T22:53:19.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.123+0000 7f2648e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f26441597d0 0x7f264406c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:19.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.123+0000 7f2648e00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f26440c7b00 0x7f26440c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:19.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.123+0000 7f2648e00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26440c0500 con 0x7f2644153c10 2024-01-21T22:53:19.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.123+0000 7f2648e00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26440c78e0 con 0x7f26440c7b00 2024-01-21T22:53:19.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.123+0000 7f2648e00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f264415a770 con 0x7f26441597d0 2024-01-21T22:53:19.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.124+0000 7f263a800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f26441597d0 0x7f264406c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:19.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.124+0000 7f263a800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f26441597d0 0x7f264406c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:44270/0 (socket says 172.21.15.138:44270) 2024-01-21T22:53:19.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.124+0000 7f263a800640 1 -- 172.21.15.138:0/2316762164 learned_addr learned my addr 172.21.15.138:0/2316762164 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:53:19.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.124+0000 7f2643400640 1 --1- 172.21.15.138:0/2316762164 >> v1:172.21.15.152:6789/0 conn(0x7f2644153c10 0x7f26440c71d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:45022/0 (socket says 172.21.15.138:45022) 2024-01-21T22:53:19.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.125+0000 7f2643e00640 1 --2- 172.21.15.138:0/2316762164 >> v2:172.21.15.152:3300/0 conn(0x7f26440c7b00 0x7f26440c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:19.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.127+0000 7f263a800640 1 -- 172.21.15.138:0/2316762164 >> v2:172.21.15.152:3300/0 conn(0x7f26440c7b00 msgr2=0x7f26440c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:53:19.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.127+0000 7f263a800640 1 --2- 172.21.15.138:0/2316762164 >> v2:172.21.15.152:3300/0 conn(0x7f26440c7b00 0x7f26440c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:19.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.127+0000 7f263a800640 1 -- 172.21.15.138:0/2316762164 >> v1:172.21.15.152:6789/0 conn(0x7f2644153c10 legacy=0x7f26440c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:19.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.127+0000 7f263a800640 1 -- 172.21.15.138:0/2316762164 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2644159ed0 con 0x7f26441597d0 2024-01-21T22:53:19.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.130+0000 7f263a800640 1 --2- 172.21.15.138:0/2316762164 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f26441597d0 0x7f264406c790 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f262c009870 tx=0x7f262c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5eccebb24413109 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:19.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.134+0000 7f2642a00640 1 -- 172.21.15.138:0/2316762164 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c030840 con 0x7f26441597d0 2024-01-21T22:53:19.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.134+0000 7f2642a00640 1 -- 172.21.15.138:0/2316762164 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f262c0309a0 con 0x7f26441597d0 2024-01-21T22:53:19.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.134+0000 7f2642a00640 1 -- 172.21.15.138:0/2316762164 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c030d50 con 0x7f26441597d0 2024-01-21T22:53:19.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.135+0000 7f2648e00640 1 -- 172.21.15.138:0/2316762164 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f26441597d0 msgr2=0x7f264406c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:19.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.135+0000 7f2648e00640 1 --2- 172.21.15.138:0/2316762164 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f26441597d0 0x7f264406c790 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f262c009870 tx=0x7f262c02b980 comp rx=0 tx=0).stop 2024-01-21T22:53:19.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.135+0000 7f2648e00640 1 -- 172.21.15.138:0/2316762164 shutdown_connections 2024-01-21T22:53:19.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.135+0000 7f2648e00640 1 --2- 172.21.15.138:0/2316762164 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f26441597d0 0x7f264406c790 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:19.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.135+0000 7f2648e00640 1 --2- 172.21.15.138:0/2316762164 >> v2:172.21.15.152:3300/0 conn(0x7f26440c7b00 0x7f26440c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:19.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.135+0000 7f2648e00640 1 -- 172.21.15.138:0/2316762164 >> 172.21.15.138:0/2316762164 conn(0x7f264406dcb0 msgr2=0x7f264406e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:53:19.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.135+0000 7f2648e00640 1 -- 172.21.15.138:0/2316762164 shutdown_connections 2024-01-21T22:53:19.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.135+0000 7f2648e00640 1 -- 172.21.15.138:0/2316762164 wait complete. 2024-01-21T22:53:19.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.137+0000 7f2648e00640 1 Processor -- start 2024-01-21T22:53:19.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.137+0000 7f2648e00640 1 -- start start 2024-01-21T22:53:19.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.138+0000 7f2648e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26440c7b00 0x7f264415c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:19.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.138+0000 7f2648e00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2644153c10 0x7f264414c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:19.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.138+0000 7f2648e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26441597d0 0x7f2644146740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:19.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.138+0000 7f2648e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264415a970 con 0x7f26441597d0 2024-01-21T22:53:19.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.138+0000 7f2648e00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2644159490 con 0x7f26440c7b00 2024-01-21T22:53:19.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.138+0000 7f2648e00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2644147430 con 0x7f2644153c10 2024-01-21T22:53:19.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.139+0000 7f2643e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26441597d0 0x7f2644146740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:19.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.139+0000 7f263a800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2644153c10 0x7f264414c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:19.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.139+0000 7f2643400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26440c7b00 0x7f264415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:19.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.139+0000 7f2643400640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26440c7b00 0x7f264415c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3300/0 says I am v2:172.21.15.138:44680/0 (socket says 172.21.15.138:44680) 2024-01-21T22:53:19.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.139+0000 7f263a800640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2644153c10 0x7f264414c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:44280/0 (socket says 172.21.15.138:44280) 2024-01-21T22:53:19.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.139+0000 7f2643400640 1 -- 172.21.15.138:0/143503167 learned_addr learned my addr 172.21.15.138:0/143503167 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:53:19.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.143+0000 7f263a800640 1 -- 172.21.15.138:0/143503167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26440c7b00 msgr2=0x7f264415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:19.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.143+0000 7f263a800640 1 --2- 172.21.15.138:0/143503167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26440c7b00 0x7f264415c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:19.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.143+0000 7f263a800640 1 -- 172.21.15.138:0/143503167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26441597d0 msgr2=0x7f2644146740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:19.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.143+0000 7f263a800640 1 --2- 172.21.15.138:0/143503167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26441597d0 0x7f2644146740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:19.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.143+0000 7f263a800640 1 -- 172.21.15.138:0/143503167 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262c0094d0 con 0x7f2644153c10 2024-01-21T22:53:19.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.143+0000 7f2643400640 1 --2- 172.21.15.138:0/143503167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26440c7b00 0x7f264415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-01-21T22:53:19.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.146+0000 7f263a800640 1 --2- 172.21.15.138:0/143503167 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2644153c10 0x7f264414c2c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f262c033000 tx=0x7f262c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:19.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.150+0000 7f263be00640 1 -- 172.21.15.138:0/143503167 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c039b50 con 0x7f2644153c10 2024-01-21T22:53:19.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.150+0000 7f263be00640 1 -- 172.21.15.138:0/143503167 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f262c039cf0 con 0x7f2644153c10 2024-01-21T22:53:19.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.150+0000 7f263be00640 1 -- 172.21.15.138:0/143503167 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c043670 con 0x7f2644153c10 2024-01-21T22:53:19.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.150+0000 7f2648e00640 1 -- 172.21.15.138:0/143503167 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26440ca000 con 0x7f2644153c10 2024-01-21T22:53:19.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.150+0000 7f2648e00640 1 -- 172.21.15.138:0/143503167 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f264415c980 con 0x7f2644153c10 2024-01-21T22:53:19.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.151+0000 7f2648e00640 1 -- 172.21.15.138:0/143503167 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f264404ea30 con 0x7f2644153c10 2024-01-21T22:53:19.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.173+0000 7f263be00640 1 -- 172.21.15.138:0/143503167 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f262c0437d0 con 0x7f2644153c10 2024-01-21T22:53:19.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.174+0000 7f263be00640 1 --2- 172.21.15.138:0/143503167 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f261c06b380 0x7f261c06d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:19.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.174+0000 7f2643400640 1 --2- 172.21.15.138:0/143503167 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f261c06b380 0x7f261c06d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:19.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.175+0000 7f2643400640 1 --2- 172.21.15.138:0/143503167 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f261c06b380 0x7f261c06d840 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2634004e00 tx=0x7f2634007f10 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:19.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.398+0000 7f263be00640 1 -- 172.21.15.138:0/143503167 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f262c07aff0 con 0x7f2644153c10 2024-01-21T22:53:19.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.409+0000 7f263be00640 1 -- 172.21.15.138:0/143503167 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f262c077330 con 0x7f2644153c10 2024-01-21T22:53:19.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.634+0000 7f2648e00640 1 -- 172.21.15.138:0/143503167 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26440ca980 con 0x7f2644153c10 2024-01-21T22:53:19.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.671+0000 7f263be00640 1 -- 172.21.15.138:0/143503167 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+16570 (secure 0 0 0) 0x7f262c039e60 con 0x7f2644153c10 2024-01-21T22:53:19.673 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-01-21T22:53:19.673 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":25,"fsid":"d488d446-149f-48d0-89ca-b2830ae72846","created":"2024-01-21T22:24:28.581932+0000","modified":"2024-01-21T22:25:37.558325+0000","last_up_change":"2024-01-21T22:24:47.145799+0000","last_in_change":"2024-01-21T22:24:37.672807+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-01-21T22:24:49.586178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-01-21T22:25:02.044454+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-01-21T22:25:03.361765+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d812b2d6-c8a3-4015-bfdb-5766b25eae5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6825","nonce":2849203277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6827","nonce":2849203277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6831","nonce":2849203277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":2849203277},{"type":"v1","addr":"172.21.15.138:6829","nonce":2849203277}]},"public_addr":"172.21.15.138:6825/2849203277","cluster_addr":"172.21.15.138:6827/2849203277","heartbeat_back_addr":"172.21.15.138:6831/2849203277","heartbeat_front_addr":"172.21.15.138:6829/2849203277","state":["exists","up"]},{"osd":1,"uuid":"d8d875df-c840-46cc-b6c0-a847f9c93999","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6817","nonce":174521731}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6819","nonce":174521731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6823","nonce":174521731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":174521731},{"type":"v1","addr":"172.21.15.138:6821","nonce":174521731}]},"public_addr":"172.21.15.138:6817/174521731","cluster_addr":"172.21.15.138:6819/174521731","heartbeat_back_addr":"172.21.15.138:6823/174521731","heartbeat_front_addr":"172.21.15.138:6821/174521731","state":["exists","up"]},{"osd":2,"uuid":"4bd94373-76ca-47a6-870b-16d96d7c9e81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6801","nonce":1365838748}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6803","nonce":1365838748}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6807","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6809","nonce":1365838748}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1365838748},{"type":"v1","addr":"172.21.15.138:6805","nonce":1365838748}]},"public_addr":"172.21.15.138:6801/1365838748","cluster_addr":"172.21.15.138:6803/1365838748","heartbeat_back_addr":"172.21.15.138:6809/1365838748","heartbeat_front_addr":"172.21.15.138:6805/1365838748","state":["exists","up"]},{"osd":3,"uuid":"07267087-d4d2-4e88-ab9b-bb87f6dd061a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6808","nonce":1841873885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6811","nonce":1841873885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6815","nonce":1841873885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1841873885},{"type":"v1","addr":"172.21.15.138:6813","nonce":1841873885}]},"public_addr":"172.21.15.138:6808/1841873885","cluster_addr":"172.21.15.138:6811/1841873885","heartbeat_back_addr":"172.21.15.138:6815/1841873885","heartbeat_front_addr":"172.21.15.138:6813/1841873885","state":["exists","up"]},{"osd":4,"uuid":"b47ef758-d783-4168-a9a5-11db3de4ca12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6826","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6827","nonce":3960812298}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6828","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6829","nonce":3960812298}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6832","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6833","nonce":3960812298}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6830","nonce":3960812298},{"type":"v1","addr":"172.21.15.152:6831","nonce":3960812298}]},"public_addr":"172.21.15.152:6827/3960812298","cluster_addr":"172.21.15.152:6829/3960812298","heartbeat_back_addr":"172.21.15.152:6833/3960812298","heartbeat_front_addr":"172.21.15.152:6831/3960812298","state":["exists","up"]},{"osd":5,"uuid":"0ea5d43e-61d8-4823-815f-9e42c62fae97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6818","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6819","nonce":4095234314}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6820","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6821","nonce":4095234314}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6824","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6825","nonce":4095234314}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6822","nonce":4095234314},{"type":"v1","addr":"172.21.15.152:6823","nonce":4095234314}]},"public_addr":"172.21.15.152:6819/4095234314","cluster_addr":"172.21.15.152:6821/4095234314","heartbeat_back_addr":"172.21.15.152:6825/4095234314","heartbeat_front_addr":"172.21.15.152:6823/4095234314","state":["exists","up"]},{"osd":6,"uuid":"59ffac36-e2e0-4693-ac82-ad5eccf6224d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6802","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6803","nonce":223848127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6804","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6805","nonce":223848127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6808","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6809","nonce":223848127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6806","nonce":223848127},{"type":"v1","addr":"172.21.15.152:6807","nonce":223848127}]},"public_addr":"172.21.15.152:6803/223848127","cluster_addr":"172.21.15.152:6805/223848127","heartbeat_back_addr":"172.21.15.152:6809/223848127","heartbeat_front_addr":"172.21.15.152:6807/223848127","state":["exists","up"]},{"osd":7,"uuid":"4f892c7d-32a5-4960-bf88-4ed7c863cd0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":24,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6810","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6811","nonce":52509095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6812","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6813","nonce":52509095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6816","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6817","nonce":52509095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.152:6814","nonce":52509095},{"type":"v1","addr":"172.21.15.152:6815","nonce":52509095}]},"public_addr":"172.21.15.152:6811/52509095","cluster_addr":"172.21.15.152:6813/52509095","heartbeat_back_addr":"172.21.15.152:6817/52509095","heartbeat_front_addr":"172.21.15.152:6815/52509095","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.434460+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.374709+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.613134+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.298148+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.242144+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.198102+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.093385+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-01-21T22:24:43.197180+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.f","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-01-21T22:53:19.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.674+0000 7f2648e00640 1 -- 172.21.15.138:0/143503167 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f261c06b380 msgr2=0x7f261c06d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:19.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.674+0000 7f2648e00640 1 --2- 172.21.15.138:0/143503167 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f261c06b380 0x7f261c06d840 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2634004e00 tx=0x7f2634007f10 comp rx=0 tx=0).stop 2024-01-21T22:53:19.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.674+0000 7f2648e00640 1 -- 172.21.15.138:0/143503167 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2644153c10 msgr2=0x7f264414c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:19.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.674+0000 7f2648e00640 1 --2- 172.21.15.138:0/143503167 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2644153c10 0x7f264414c2c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f262c033000 tx=0x7f262c004730 comp rx=0 tx=0).stop 2024-01-21T22:53:19.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.674+0000 7f2648e00640 1 -- 172.21.15.138:0/143503167 shutdown_connections 2024-01-21T22:53:19.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.674+0000 7f2648e00640 1 --2- 172.21.15.138:0/143503167 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f261c06b380 0x7f261c06d840 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:19.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.674+0000 7f2648e00640 1 --2- 172.21.15.138:0/143503167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26441597d0 0x7f2644146740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:19.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.674+0000 7f2648e00640 1 --2- 172.21.15.138:0/143503167 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f2644153c10 0x7f264414c2c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:19.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.674+0000 7f2648e00640 1 --2- 172.21.15.138:0/143503167 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f26440c7b00 0x7f264415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:19.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.674+0000 7f2648e00640 1 -- 172.21.15.138:0/143503167 >> 172.21.15.138:0/143503167 conn(0x7f264406dcb0 msgr2=0x7f2644151310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:53:19.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.674+0000 7f2648e00640 1 -- 172.21.15.138:0/143503167 shutdown_connections 2024-01-21T22:53:19.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.674+0000 7f2648e00640 1 -- 172.21.15.138:0/143503167 wait complete. 2024-01-21T22:53:19.696 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-01-21T22:53:19.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.896+0000 7f398bc00640 1 Processor -- start 2024-01-21T22:53:19.898 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.897+0000 7f398bc00640 1 -- start start 2024-01-21T22:53:19.899 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.898+0000 7f398bc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f39840c6c40 0x7f39840703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:19.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.898+0000 7f398bc00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3984070ec0 0x7f3984071330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:19.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.898+0000 7f398bc00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39841506a0 con 0x7f39841502c0 2024-01-21T22:53:19.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.898+0000 7f398bc00640 1 -- --> v2:172.21.15.138:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3984071870 con 0x7f39840c6c40 2024-01-21T22:53:19.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.898+0000 7f398bc00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39840c7080 con 0x7f3984070ec0 2024-01-21T22:53:19.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.899+0000 7f397bc00640 1 --2- >> v2:172.21.15.138:3300/0 conn(0x7f39840c6c40 0x7f39840703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:19.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.899+0000 7f3989200640 1 --1- >> v1:172.21.15.152:6789/0 conn(0x7f39841502c0 0x7f398406cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.152:6789/0 says I am v1:172.21.15.138:45038/0 (socket says 172.21.15.138:45038) 2024-01-21T22:53:19.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.899+0000 7f3989200640 1 -- 172.21.15.138:0/4154076172 learned_addr learned my addr 172.21.15.138:0/4154076172 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:53:19.901 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.899+0000 7f3989c00640 1 --2- 172.21.15.138:0/4154076172 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3984070ec0 0x7f3984071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:19.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.901+0000 7f3989c00640 1 -- 172.21.15.138:0/4154076172 >> v2:172.21.15.138:3300/0 conn(0x7f39840c6c40 msgr2=0x7f39840703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:53:19.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.901+0000 7f3989c00640 1 --2- 172.21.15.138:0/4154076172 >> v2:172.21.15.138:3300/0 conn(0x7f39840c6c40 0x7f39840703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:19.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.901+0000 7f3989c00640 1 -- 172.21.15.138:0/4154076172 >> v1:172.21.15.152:6789/0 conn(0x7f39841502c0 legacy=0x7f398406cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:19.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.901+0000 7f3989c00640 1 -- 172.21.15.138:0/4154076172 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39840c76d0 con 0x7f3984070ec0 2024-01-21T22:53:19.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.903+0000 7f3989c00640 1 --2- 172.21.15.138:0/4154076172 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3984070ec0 0x7f3984071330 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f397c0097a0 tx=0x7f397c02b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6144be5a4ba8eb server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:19.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.907+0000 7f3983e00640 1 -- 172.21.15.138:0/4154076172 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c030610 con 0x7f3984070ec0 2024-01-21T22:53:19.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.907+0000 7f3983e00640 1 -- 172.21.15.138:0/4154076172 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f397c030770 con 0x7f3984070ec0 2024-01-21T22:53:19.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.907+0000 7f3983e00640 1 -- 172.21.15.138:0/4154076172 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c030b20 con 0x7f3984070ec0 2024-01-21T22:53:19.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.908+0000 7f398bc00640 1 -- 172.21.15.138:0/4154076172 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3984070ec0 msgr2=0x7f3984071330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:19.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.908+0000 7f398bc00640 1 --2- 172.21.15.138:0/4154076172 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3984070ec0 0x7f3984071330 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f397c0097a0 tx=0x7f397c02b960 comp rx=0 tx=0).stop 2024-01-21T22:53:19.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.908+0000 7f398bc00640 1 -- 172.21.15.138:0/4154076172 shutdown_connections 2024-01-21T22:53:19.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.908+0000 7f398bc00640 1 --2- 172.21.15.138:0/4154076172 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f3984070ec0 0x7f3984071330 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:19.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.908+0000 7f398bc00640 1 --2- 172.21.15.138:0/4154076172 >> v2:172.21.15.138:3300/0 conn(0x7f39840c6c40 0x7f39840703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:19.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.908+0000 7f398bc00640 1 -- 172.21.15.138:0/4154076172 >> 172.21.15.138:0/4154076172 conn(0x7f39840cb070 msgr2=0x7f39840cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:53:19.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.908+0000 7f398bc00640 1 -- 172.21.15.138:0/4154076172 shutdown_connections 2024-01-21T22:53:19.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.908+0000 7f398bc00640 1 -- 172.21.15.138:0/4154076172 wait complete. 2024-01-21T22:53:19.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.910+0000 7f398bc00640 1 Processor -- start 2024-01-21T22:53:19.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.910+0000 7f398bc00640 1 -- start start 2024-01-21T22:53:19.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.911+0000 7f398bc00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3984070ec0 0x7f39841f3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:19.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.911+0000 7f398bc00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39840c6c40 0x7f3984140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:19.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.911+0000 7f398bc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39841502c0 0x7f3984140a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:19.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.911+0000 7f398bc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39840bf070 con 0x7f39841502c0 2024-01-21T22:53:19.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.911+0000 7f398bc00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3984156e40 con 0x7f3984070ec0 2024-01-21T22:53:19.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.911+0000 7f398bc00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3984156ba0 con 0x7f39840c6c40 2024-01-21T22:53:19.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.912+0000 7f3989c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39841502c0 0x7f3984140a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:19.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.912+0000 7f397bc00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39840c6c40 0x7f3984140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:19.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.912+0000 7f3989200640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3984070ec0 0x7f39841f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:19.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.912+0000 7f3989c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39841502c0 0x7f3984140a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42332/0 (socket says 172.21.15.138:42332) 2024-01-21T22:53:19.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.912+0000 7f3989c00640 1 -- 172.21.15.138:0/2886967455 learned_addr learned my addr 172.21.15.138:0/2886967455 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:53:19.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.916+0000 7f397bc00640 1 -- 172.21.15.138:0/2886967455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3984070ec0 msgr2=0x7f39841f3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:19.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.916+0000 7f397bc00640 1 --2- 172.21.15.138:0/2886967455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3984070ec0 0x7f39841f3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:19.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.916+0000 7f397bc00640 1 -- 172.21.15.138:0/2886967455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39841502c0 msgr2=0x7f3984140a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:19.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.916+0000 7f397bc00640 1 --2- 172.21.15.138:0/2886967455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39841502c0 0x7f3984140a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:19.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.916+0000 7f397bc00640 1 -- 172.21.15.138:0/2886967455 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c009400 con 0x7f39840c6c40 2024-01-21T22:53:19.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.918+0000 7f397bc00640 1 --2- 172.21.15.138:0/2886967455 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39840c6c40 0x7f3984140530 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f396c00e8a0 tx=0x7f396c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:19.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.921+0000 7f3981600640 1 -- 172.21.15.138:0/2886967455 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c009e60 con 0x7f39840c6c40 2024-01-21T22:53:19.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.922+0000 7f3981600640 1 -- 172.21.15.138:0/2886967455 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f396c00c940 con 0x7f39840c6c40 2024-01-21T22:53:19.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.922+0000 7f3981600640 1 -- 172.21.15.138:0/2886967455 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f396c00ccf0 con 0x7f39840c6c40 2024-01-21T22:53:19.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.922+0000 7f398bc00640 1 -- 172.21.15.138:0/2886967455 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39840c3590 con 0x7f39840c6c40 2024-01-21T22:53:19.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.922+0000 7f398bc00640 1 -- 172.21.15.138:0/2886967455 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39840c0d90 con 0x7f39840c6c40 2024-01-21T22:53:19.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.923+0000 7f398bc00640 1 -- 172.21.15.138:0/2886967455 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39840c0fb0 con 0x7f39840c6c40 2024-01-21T22:53:19.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.942+0000 7f3981600640 1 -- 172.21.15.138:0/2886967455 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f396c013470 con 0x7f39840c6c40 2024-01-21T22:53:19.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.943+0000 7f3981600640 1 --2- 172.21.15.138:0/2886967455 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f395c06b3b0 0x7f395c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:19.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.944+0000 7f3989200640 1 --2- 172.21.15.138:0/2886967455 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f395c06b3b0 0x7f395c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:19.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.944+0000 7f3981600640 1 -- 172.21.15.138:0/2886967455 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f396c058d30 con 0x7f39840c6c40 2024-01-21T22:53:19.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:19.944+0000 7f3989200640 1 --2- 172.21.15.138:0/2886967455 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f395c06b3b0 0x7f395c06d870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3974005c00 tx=0x7f3974009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:20.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:20.180+0000 7f3981600640 1 -- 172.21.15.138:0/2886967455 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f396c054fe0 con 0x7f39840c6c40 2024-01-21T22:53:20.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:20.442+0000 7f398bc00640 1 -- 172.21.15.138:0/2886967455 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f398404ea30 con 0x7f39840c6c40 2024-01-21T22:53:20.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:20.915+0000 7f3981600640 1 -- 172.21.15.138:0/2886967455 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v11) v1 ==== 161+0+0 (secure 0 0 0) 0x7f396c021020 con 0x7f39840c6c40 2024-01-21T22:53:20.917 INFO:teuthology.orchestra.run.smithi138.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-01-21T22:53:20.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:20.920+0000 7f398bc00640 1 -- 172.21.15.138:0/2886967455 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f395c06b3b0 msgr2=0x7f395c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:20.920+0000 7f398bc00640 1 --2- 172.21.15.138:0/2886967455 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f395c06b3b0 0x7f395c06d870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3974005c00 tx=0x7f3974009040 comp rx=0 tx=0).stop 2024-01-21T22:53:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:20.920+0000 7f398bc00640 1 -- 172.21.15.138:0/2886967455 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39840c6c40 msgr2=0x7f3984140530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:20.920+0000 7f398bc00640 1 --2- 172.21.15.138:0/2886967455 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39840c6c40 0x7f3984140530 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f396c00e8a0 tx=0x7f396c00ed60 comp rx=0 tx=0).stop 2024-01-21T22:53:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:20.920+0000 7f398bc00640 1 -- 172.21.15.138:0/2886967455 shutdown_connections 2024-01-21T22:53:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:20.920+0000 7f398bc00640 1 --2- 172.21.15.138:0/2886967455 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f395c06b3b0 0x7f395c06d870 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:20.920+0000 7f398bc00640 1 --2- 172.21.15.138:0/2886967455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39841502c0 0x7f3984140a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:20.920+0000 7f398bc00640 1 --2- 172.21.15.138:0/2886967455 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f39840c6c40 0x7f3984140530 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:20.920+0000 7f398bc00640 1 --2- 172.21.15.138:0/2886967455 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f3984070ec0 0x7f39841f3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:20.920+0000 7f398bc00640 1 -- 172.21.15.138:0/2886967455 >> 172.21.15.138:0/2886967455 conn(0x7f39840cb070 msgr2=0x7f398406a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:53:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:20.920+0000 7f398bc00640 1 -- 172.21.15.138:0/2886967455 shutdown_connections 2024-01-21T22:53:20.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:20.920+0000 7f398bc00640 1 -- 172.21.15.138:0/2886967455 wait complete. 2024-01-21T22:53:20.940 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-01-21T22:53:21.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.137+0000 7f7bffe00640 1 Processor -- start 2024-01-21T22:53:21.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.137+0000 7f7bffe00640 1 -- start start 2024-01-21T22:53:21.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.138+0000 7f7bffe00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7bf8151a30 0x7f7bf8157730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:21.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.138+0000 7f7bffe00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7bf8071bb0 0x7f7bf81620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:21.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.138+0000 7f7bffe00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bf8070590 con 0x7f7bf806b4f0 2024-01-21T22:53:21.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.138+0000 7f7bffe00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf806ac50 con 0x7f7bf8151a30 2024-01-21T22:53:21.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.138+0000 7f7bffe00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf806a6f0 con 0x7f7bf8071bb0 2024-01-21T22:53:21.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.139+0000 7f7bfd400640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7f7bf8151a30 0x7f7bf8157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:21.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.139+0000 7f7bfde00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7bf8071bb0 0x7f7bf81620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:21.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.139+0000 7f7bfca00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f7bf806b4f0 0x7f7bf80c4750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56906/0 (socket says 172.21.15.138:56906) 2024-01-21T22:53:21.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.139+0000 7f7bfca00640 1 -- 172.21.15.138:0/1202705380 learned_addr learned my addr 172.21.15.138:0/1202705380 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:53:21.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.142+0000 7f7bfde00640 1 -- 172.21.15.138:0/1202705380 >> v2:172.21.15.152:3300/0 conn(0x7f7bf8151a30 msgr2=0x7f7bf8157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:53:21.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.142+0000 7f7bfde00640 1 --2- 172.21.15.138:0/1202705380 >> v2:172.21.15.152:3300/0 conn(0x7f7bf8151a30 0x7f7bf8157730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:21.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.142+0000 7f7bfde00640 1 -- 172.21.15.138:0/1202705380 >> v1:172.21.15.138:6789/0 conn(0x7f7bf806b4f0 legacy=0x7f7bf80c4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:21.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.142+0000 7f7bfde00640 1 -- 172.21.15.138:0/1202705380 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf8151e10 con 0x7f7bf8071bb0 2024-01-21T22:53:21.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.144+0000 7f7bfde00640 1 --2- 172.21.15.138:0/1202705380 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7bf8071bb0 0x7f7bf81620b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0009d90 tx=0x7f7bf002bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=820128302be87ec5 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:21.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.148+0000 7f7bf7e00640 1 -- 172.21.15.138:0/1202705380 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf002cce0 con 0x7f7bf8071bb0 2024-01-21T22:53:21.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.148+0000 7f7bf7e00640 1 -- 172.21.15.138:0/1202705380 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7bf002ce40 con 0x7f7bf8071bb0 2024-01-21T22:53:21.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.148+0000 7f7bf7e00640 1 -- 172.21.15.138:0/1202705380 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0034680 con 0x7f7bf8071bb0 2024-01-21T22:53:21.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.149+0000 7f7bffe00640 1 -- 172.21.15.138:0/1202705380 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7bf8071bb0 msgr2=0x7f7bf81620b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:21.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.149+0000 7f7bffe00640 1 --2- 172.21.15.138:0/1202705380 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7bf8071bb0 0x7f7bf81620b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0009d90 tx=0x7f7bf002bf30 comp rx=0 tx=0).stop 2024-01-21T22:53:21.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.149+0000 7f7bffe00640 1 -- 172.21.15.138:0/1202705380 shutdown_connections 2024-01-21T22:53:21.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.149+0000 7f7bffe00640 1 --2- 172.21.15.138:0/1202705380 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7bf8071bb0 0x7f7bf81620b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:21.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.149+0000 7f7bffe00640 1 --2- 172.21.15.138:0/1202705380 >> v2:172.21.15.152:3300/0 conn(0x7f7bf8151a30 0x7f7bf8157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:21.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.149+0000 7f7bffe00640 1 -- 172.21.15.138:0/1202705380 >> 172.21.15.138:0/1202705380 conn(0x7f7bf8072a20 msgr2=0x7f7bf8066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:53:21.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.149+0000 7f7bffe00640 1 -- 172.21.15.138:0/1202705380 shutdown_connections 2024-01-21T22:53:21.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.149+0000 7f7bffe00640 1 -- 172.21.15.138:0/1202705380 wait complete. 2024-01-21T22:53:21.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.150+0000 7f7bffe00640 1 Processor -- start 2024-01-21T22:53:21.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.151+0000 7f7bffe00640 1 -- start start 2024-01-21T22:53:21.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.151+0000 7f7bffe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bf806b4f0 0x7f7bf813d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:21.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.151+0000 7f7bffe00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7bf8071bb0 0x7f7bf8134c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:21.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.151+0000 7f7bffe00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7bf8151a30 0x7f7bf81ee4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:21.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.151+0000 7f7bffe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf81ee050 con 0x7f7bf806b4f0 2024-01-21T22:53:21.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.152+0000 7f7bffe00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf81f2270 con 0x7f7bf8151a30 2024-01-21T22:53:21.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.152+0000 7f7bffe00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf813dee0 con 0x7f7bf8071bb0 2024-01-21T22:53:21.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.152+0000 7f7bfca00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7bf8071bb0 0x7f7bf8134c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:21.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.152+0000 7f7bfd400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bf806b4f0 0x7f7bf813d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:21.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.152+0000 7f7bfde00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7bf8151a30 0x7f7bf81ee4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:21.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.152+0000 7f7bfca00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7bf8071bb0 0x7f7bf8134c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:44322/0 (socket says 172.21.15.138:44322) 2024-01-21T22:53:21.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.152+0000 7f7bfca00640 1 -- 172.21.15.138:0/3560138611 learned_addr learned my addr 172.21.15.138:0/3560138611 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:53:21.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.156+0000 7f7bfca00640 1 -- 172.21.15.138:0/3560138611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7bf8151a30 msgr2=0x7f7bf81ee4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:21.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.156+0000 7f7bfca00640 1 --2- 172.21.15.138:0/3560138611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7bf8151a30 0x7f7bf81ee4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:21.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.156+0000 7f7bfca00640 1 -- 172.21.15.138:0/3560138611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bf806b4f0 msgr2=0x7f7bf813d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:21.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.156+0000 7f7bfca00640 1 --2- 172.21.15.138:0/3560138611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bf806b4f0 0x7f7bf813d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:21.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.156+0000 7f7bfca00640 1 -- 172.21.15.138:0/3560138611 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf0009a70 con 0x7f7bf8071bb0 2024-01-21T22:53:21.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.158+0000 7f7bfca00640 1 --2- 172.21.15.138:0/3560138611 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7bf8071bb0 0x7f7bf8134c40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7be000c400 tx=0x7f7be000c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:21.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.162+0000 7f7bf5600640 1 -- 172.21.15.138:0/3560138611 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be0005ce0 con 0x7f7bf8071bb0 2024-01-21T22:53:21.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.162+0000 7f7bf5600640 1 -- 172.21.15.138:0/3560138611 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7be0005e40 con 0x7f7bf8071bb0 2024-01-21T22:53:21.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.162+0000 7f7bf5600640 1 -- 172.21.15.138:0/3560138611 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be0007330 con 0x7f7bf8071bb0 2024-01-21T22:53:21.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.162+0000 7f7bffe00640 1 -- 172.21.15.138:0/3560138611 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf81f16f0 con 0x7f7bf8071bb0 2024-01-21T22:53:21.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.162+0000 7f7bffe00640 1 -- 172.21.15.138:0/3560138611 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf81f1cb0 con 0x7f7bf8071bb0 2024-01-21T22:53:21.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.164+0000 7f7bffe00640 1 -- 172.21.15.138:0/3560138611 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf81ede20 con 0x7f7bf8071bb0 2024-01-21T22:53:21.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.186+0000 7f7bf5600640 1 -- 172.21.15.138:0/3560138611 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f7be0007490 con 0x7f7bf8071bb0 2024-01-21T22:53:21.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.187+0000 7f7bf5600640 1 --2- 172.21.15.138:0/3560138611 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7bd006b3b0 0x7f7bd006d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:21.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.188+0000 7f7bfd400640 1 --2- 172.21.15.138:0/3560138611 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7bd006b3b0 0x7f7bd006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:21.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.188+0000 7f7bfd400640 1 --2- 172.21.15.138:0/3560138611 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7bd006b3b0 0x7f7bd006d870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009aa0 tx=0x7f7be8009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:21.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.419+0000 7f7bf5600640 1 -- 172.21.15.138:0/3560138611 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f7be0058020 con 0x7f7bf8071bb0 2024-01-21T22:53:21.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.430+0000 7f7bf5600640 1 -- 172.21.15.138:0/3560138611 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f7be00542d0 con 0x7f7bf8071bb0 2024-01-21T22:53:21.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:21.672+0000 7f7bffe00640 1 -- 172.21.15.138:0/3560138611 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7bf814bab0 con 0x7f7bf8071bb0 2024-01-21T22:53:22.014 INFO:tasks.ceph.mds.d.smithi152.stderr:==00:00:28:33.558 43351== 2024-01-21T22:53:22.014 INFO:tasks.ceph.mds.d.smithi152.stderr:==00:00:28:33.558 43351== Exit program on first error (--exit-on-first-error=yes) 2024-01-21T22:53:22.184 INFO:tasks.ceph.mds.d.smithi152.stderr:daemon-helper: command failed with exit status 42 2024-01-21T22:53:22.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:22.985+0000 7f7bf5600640 1 -- 172.21.15.138:0/3560138611 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v12) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7be00170a0 con 0x7f7bf8071bb0 2024-01-21T22:53:22.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:22.988+0000 7f7bffe00640 1 -- 172.21.15.138:0/3560138611 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7bd006b3b0 msgr2=0x7f7bd006d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:22.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:22.988+0000 7f7bffe00640 1 --2- 172.21.15.138:0/3560138611 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7bd006b3b0 0x7f7bd006d870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009aa0 tx=0x7f7be8009190 comp rx=0 tx=0).stop 2024-01-21T22:53:22.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:22.989+0000 7f7bffe00640 1 -- 172.21.15.138:0/3560138611 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7bf8071bb0 msgr2=0x7f7bf8134c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:22.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:22.989+0000 7f7bffe00640 1 --2- 172.21.15.138:0/3560138611 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7bf8071bb0 0x7f7bf8134c40 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7be000c400 tx=0x7f7be000c8c0 comp rx=0 tx=0).stop 2024-01-21T22:53:22.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:22.989+0000 7f7bffe00640 1 -- 172.21.15.138:0/3560138611 shutdown_connections 2024-01-21T22:53:22.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:22.989+0000 7f7bffe00640 1 --2- 172.21.15.138:0/3560138611 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f7bd006b3b0 0x7f7bd006d870 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:22.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:22.989+0000 7f7bffe00640 1 --2- 172.21.15.138:0/3560138611 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f7bf8151a30 0x7f7bf81ee4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:22.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:22.989+0000 7f7bffe00640 1 --2- 172.21.15.138:0/3560138611 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f7bf8071bb0 0x7f7bf8134c40 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:22.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:22.989+0000 7f7bffe00640 1 --2- 172.21.15.138:0/3560138611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bf806b4f0 0x7f7bf813d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:22.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:22.989+0000 7f7bffe00640 1 -- 172.21.15.138:0/3560138611 >> 172.21.15.138:0/3560138611 conn(0x7f7bf8072a20 msgr2=0x7f7bf8066400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:53:22.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:22.989+0000 7f7bffe00640 1 -- 172.21.15.138:0/3560138611 shutdown_connections 2024-01-21T22:53:22.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:22.989+0000 7f7bffe00640 1 -- 172.21.15.138:0/3560138611 wait complete. 2024-01-21T22:53:23.011 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-01-21T22:53:23.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.224+0000 7fe3cca00640 1 Processor -- start 2024-01-21T22:53:23.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.225+0000 7fe3cca00640 1 -- start start 2024-01-21T22:53:23.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.226+0000 7fe3cca00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3c406c050 0x7fe3c41568b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:23.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.226+0000 7fe3cca00640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe3c4070d30 0x7fe3c406d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:23.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.226+0000 7fe3cca00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c406d690 con 0x7fe3c406c430 2024-01-21T22:53:23.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.226+0000 7fe3cca00640 1 -- --> v2:172.21.15.152:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c40c5930 con 0x7fe3c4070d30 2024-01-21T22:53:23.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.226+0000 7fe3cca00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c4156ef0 con 0x7fe3c406c050 2024-01-21T22:53:23.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.227+0000 7fe3c9600640 1 --2- >> v2:172.21.15.152:3300/0 conn(0x7fe3c4070d30 0x7fe3c406d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:23.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.227+0000 7fe3ca000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3c406c050 0x7fe3c41568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:23.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.227+0000 7fe3ca000640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3c406c050 0x7fe3c41568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:44328/0 (socket says 172.21.15.138:44328) 2024-01-21T22:53:23.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.227+0000 7fe3ca000640 1 -- 172.21.15.138:0/165328367 learned_addr learned my addr 172.21.15.138:0/165328367 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:53:23.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.229+0000 7fe3ca000640 1 -- 172.21.15.138:0/165328367 >> v2:172.21.15.152:3300/0 conn(0x7fe3c4070d30 msgr2=0x7fe3c406d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-01-21T22:53:23.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.229+0000 7fe3ca000640 1 --2- 172.21.15.138:0/165328367 >> v2:172.21.15.152:3300/0 conn(0x7fe3c4070d30 0x7fe3c406d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:23.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.229+0000 7fe3ca000640 1 -- 172.21.15.138:0/165328367 >> v1:172.21.15.152:6789/0 conn(0x7fe3c406c430 legacy=0x7fe3c4150770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:23.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.229+0000 7fe3ca000640 1 -- 172.21.15.138:0/165328367 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c41571d0 con 0x7fe3c406c050 2024-01-21T22:53:23.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.232+0000 7fe3ca000640 1 --2- 172.21.15.138:0/165328367 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3c406c050 0x7fe3c41568b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe3b400bb10 tx=0x7fe3b402e1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69d3a81ae6361d9a server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:23.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.236+0000 7fe3c8c00640 1 -- 172.21.15.138:0/165328367 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4036440 con 0x7fe3c406c050 2024-01-21T22:53:23.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.236+0000 7fe3c8c00640 1 -- 172.21.15.138:0/165328367 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe3b40365a0 con 0x7fe3c406c050 2024-01-21T22:53:23.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.236+0000 7fe3c8c00640 1 -- 172.21.15.138:0/165328367 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4036950 con 0x7fe3c406c050 2024-01-21T22:53:23.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.237+0000 7fe3cca00640 1 -- 172.21.15.138:0/165328367 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3c406c050 msgr2=0x7fe3c41568b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:23.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.237+0000 7fe3cca00640 1 --2- 172.21.15.138:0/165328367 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3c406c050 0x7fe3c41568b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe3b400bb10 tx=0x7fe3b402e1e0 comp rx=0 tx=0).stop 2024-01-21T22:53:23.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.237+0000 7fe3cca00640 1 -- 172.21.15.138:0/165328367 shutdown_connections 2024-01-21T22:53:23.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.237+0000 7fe3cca00640 1 --2- 172.21.15.138:0/165328367 >> v2:172.21.15.152:3300/0 conn(0x7fe3c4070d30 0x7fe3c406d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:23.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.237+0000 7fe3cca00640 1 --2- 172.21.15.138:0/165328367 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3c406c050 0x7fe3c41568b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:23.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.237+0000 7fe3cca00640 1 -- 172.21.15.138:0/165328367 >> 172.21.15.138:0/165328367 conn(0x7fe3c406add0 msgr2=0x7fe3c40caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:53:23.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.237+0000 7fe3cca00640 1 -- 172.21.15.138:0/165328367 shutdown_connections 2024-01-21T22:53:23.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.237+0000 7fe3cca00640 1 -- 172.21.15.138:0/165328367 wait complete. 2024-01-21T22:53:23.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.238+0000 7fe3cca00640 1 Processor -- start 2024-01-21T22:53:23.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.239+0000 7fe3cca00640 1 -- start start 2024-01-21T22:53:23.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.239+0000 7fe3cca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3c406c050 0x7fe3c4153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:23.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.239+0000 7fe3cca00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3c406c430 0x7fe3c4151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:23.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.240+0000 7fe3cca00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3c4070d30 0x7fe3c40c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:23.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.240+0000 7fe3cca00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c40aa930 con 0x7fe3c406c050 2024-01-21T22:53:23.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.240+0000 7fe3cca00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c40c9240 con 0x7fe3c4070d30 2024-01-21T22:53:23.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.240+0000 7fe3cca00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c40c8f10 con 0x7fe3c406c430 2024-01-21T22:53:23.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.240+0000 7fe3caa00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3c4070d30 0x7fe3c40c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:23.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.240+0000 7fe3ca000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3c406c050 0x7fe3c4153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:23.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.240+0000 7fe3c9600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3c406c430 0x7fe3c4151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:23.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.241+0000 7fe3c9600640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3c406c430 0x7fe3c4151470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:44336/0 (socket says 172.21.15.138:44336) 2024-01-21T22:53:23.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.241+0000 7fe3c9600640 1 -- 172.21.15.138:0/2656409550 learned_addr learned my addr 172.21.15.138:0/2656409550 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:53:23.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.244+0000 7fe3c9600640 1 -- 172.21.15.138:0/2656409550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3c4070d30 msgr2=0x7fe3c40c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:23.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.244+0000 7fe3c9600640 1 --2- 172.21.15.138:0/2656409550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3c4070d30 0x7fe3c40c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:23.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.244+0000 7fe3c9600640 1 -- 172.21.15.138:0/2656409550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3c406c050 msgr2=0x7fe3c4153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:23.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.244+0000 7fe3c9600640 1 --2- 172.21.15.138:0/2656409550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3c406c050 0x7fe3c4153e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:23.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.244+0000 7fe3c9600640 1 -- 172.21.15.138:0/2656409550 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b400b770 con 0x7fe3c406c430 2024-01-21T22:53:23.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.247+0000 7fe3c9600640 1 --2- 172.21.15.138:0/2656409550 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3c406c430 0x7fe3c4151470 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00e870 tx=0x7fe3ac00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:23.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.250+0000 7fe3c2000640 1 -- 172.21.15.138:0/2656409550 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac0094f0 con 0x7fe3c406c430 2024-01-21T22:53:23.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.250+0000 7fe3c2000640 1 -- 172.21.15.138:0/2656409550 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe3ac00c940 con 0x7fe3c406c430 2024-01-21T22:53:23.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.250+0000 7fe3c2000640 1 -- 172.21.15.138:0/2656409550 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac00cd10 con 0x7fe3c406c430 2024-01-21T22:53:23.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.250+0000 7fe3cca00640 1 -- 172.21.15.138:0/2656409550 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c4152960 con 0x7fe3c406c430 2024-01-21T22:53:23.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.250+0000 7fe3cca00640 1 -- 172.21.15.138:0/2656409550 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c4152f20 con 0x7fe3c406c430 2024-01-21T22:53:23.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.252+0000 7fe3cca00640 1 -- 172.21.15.138:0/2656409550 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3c406caa0 con 0x7fe3c406c430 2024-01-21T22:53:23.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.272+0000 7fe3c2000640 1 -- 172.21.15.138:0/2656409550 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7fe3ac013470 con 0x7fe3c406c430 2024-01-21T22:53:23.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.273+0000 7fe3c2000640 1 --2- 172.21.15.138:0/2656409550 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe39c06b380 0x7fe39c06d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:23.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.273+0000 7fe3c2000640 1 -- 172.21.15.138:0/2656409550 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fe3ac08a3c0 con 0x7fe3c406c430 2024-01-21T22:53:23.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.274+0000 7fe3ca000640 1 --2- 172.21.15.138:0/2656409550 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe39c06b380 0x7fe39c06d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:23.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.274+0000 7fe3ca000640 1 --2- 172.21.15.138:0/2656409550 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe39c06b380 0x7fe39c06d840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4038000 tx=0x7fe3b4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:23.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.509+0000 7fe3c2000640 1 -- 172.21.15.138:0/2656409550 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7fe3ac055100 con 0x7fe3c406c430 2024-01-21T22:53:23.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:23.714+0000 7fe3cca00640 1 -- 172.21.15.138:0/2656409550 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe3c404eaa0 con 0x7fe3c406c430 2024-01-21T22:53:24.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.118+0000 7fe3c2000640 1 -- 172.21.15.138:0/2656409550 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v27) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe3ac0170d0 con 0x7fe3c406c430 2024-01-21T22:53:24.120 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_metadata' removed 2024-01-21T22:53:24.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.123+0000 7fe3cca00640 1 -- 172.21.15.138:0/2656409550 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe39c06b380 msgr2=0x7fe39c06d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:24.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.123+0000 7fe3cca00640 1 --2- 172.21.15.138:0/2656409550 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe39c06b380 0x7fe39c06d840 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4038000 tx=0x7fe3b4008040 comp rx=0 tx=0).stop 2024-01-21T22:53:24.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.123+0000 7fe3cca00640 1 -- 172.21.15.138:0/2656409550 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3c406c430 msgr2=0x7fe3c4151470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:24.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.123+0000 7fe3cca00640 1 --2- 172.21.15.138:0/2656409550 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3c406c430 0x7fe3c4151470 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00e870 tx=0x7fe3ac00ed30 comp rx=0 tx=0).stop 2024-01-21T22:53:24.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.123+0000 7fe3cca00640 1 -- 172.21.15.138:0/2656409550 shutdown_connections 2024-01-21T22:53:24.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.123+0000 7fe3cca00640 1 --2- 172.21.15.138:0/2656409550 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7fe39c06b380 0x7fe39c06d840 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:24.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.123+0000 7fe3cca00640 1 --2- 172.21.15.138:0/2656409550 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7fe3c4070d30 0x7fe3c40c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:24.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.123+0000 7fe3cca00640 1 --2- 172.21.15.138:0/2656409550 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7fe3c406c430 0x7fe3c4151470 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:24.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.123+0000 7fe3cca00640 1 --2- 172.21.15.138:0/2656409550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3c406c050 0x7fe3c4153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:24.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.123+0000 7fe3cca00640 1 -- 172.21.15.138:0/2656409550 >> 172.21.15.138:0/2656409550 conn(0x7fe3c406add0 msgr2=0x7fe3c40c7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:53:24.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.123+0000 7fe3cca00640 1 -- 172.21.15.138:0/2656409550 shutdown_connections 2024-01-21T22:53:24.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.123+0000 7fe3cca00640 1 -- 172.21.15.138:0/2656409550 wait complete. 2024-01-21T22:53:24.152 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-01-21T22:53:24.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.358+0000 7f8d96a00640 1 Processor -- start 2024-01-21T22:53:24.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.359+0000 7f8d96a00640 1 -- start start 2024-01-21T22:53:24.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.360+0000 7f8d96a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8d9006dde0 0x7f8d900cb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:24.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.360+0000 7f8d96a00640 1 -- --> v1:172.21.15.138:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d9015e3e0 con 0x7f8d90157d10 2024-01-21T22:53:24.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.360+0000 7f8d96a00640 1 -- --> v1:172.21.15.152:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d9015ef80 con 0x7f8d9006e1c0 2024-01-21T22:53:24.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.360+0000 7f8d96a00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d90161570 con 0x7f8d9006dde0 2024-01-21T22:53:24.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.361+0000 7f8d94a00640 1 --1- >> v1:172.21.15.138:6789/0 conn(0x7f8d90157d10 0x7f8d90158150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.138:6789/0 says I am v1:172.21.15.138:56912/0 (socket says 172.21.15.138:56912) 2024-01-21T22:53:24.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.361+0000 7f8d94a00640 1 -- 172.21.15.138:0/2454588535 learned_addr learned my addr 172.21.15.138:0/2454588535 (peer_addr_for_me v1:172.21.15.138:0/0) 2024-01-21T22:53:24.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.361+0000 7f8d8fe00640 1 --2- 172.21.15.138:0/2454588535 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8d9006dde0 0x7f8d900cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:24.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.363+0000 7f8d8f400640 1 -- 172.21.15.138:0/2454588535 <== mon.1 v1:172.21.15.152:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922218489 0 0) 0x7f8d9015ef80 con 0x7f8d9006e1c0 2024-01-21T22:53:24.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.364+0000 7f8d8f400640 1 -- 172.21.15.138:0/2454588535 --> v1:172.21.15.152:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d70003520 con 0x7f8d9006e1c0 2024-01-21T22:53:24.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.364+0000 7f8d8fe00640 1 -- 172.21.15.138:0/2454588535 >> v1:172.21.15.152:6789/0 conn(0x7f8d9006e1c0 legacy=0x7f8d9015e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:24.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.364+0000 7f8d8fe00640 1 -- 172.21.15.138:0/2454588535 >> v1:172.21.15.138:6789/0 conn(0x7f8d90157d10 legacy=0x7f8d90158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:24.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.364+0000 7f8d8fe00640 1 -- 172.21.15.138:0/2454588535 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d90161810 con 0x7f8d9006dde0 2024-01-21T22:53:24.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.366+0000 7f8d8fe00640 1 --2- 172.21.15.138:0/2454588535 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8d9006dde0 0x7f8d900cb830 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8d800097d0 tx=0x7f8d8002b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36c51bb0b2264fa5 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:24.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.370+0000 7f8d8f400640 1 -- 172.21.15.138:0/2454588535 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d80030880 con 0x7f8d9006dde0 2024-01-21T22:53:24.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.370+0000 7f8d8f400640 1 -- 172.21.15.138:0/2454588535 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f8d800309e0 con 0x7f8d9006dde0 2024-01-21T22:53:24.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.370+0000 7f8d8f400640 1 -- 172.21.15.138:0/2454588535 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d80030d90 con 0x7f8d9006dde0 2024-01-21T22:53:24.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.370+0000 7f8d96a00640 1 -- 172.21.15.138:0/2454588535 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8d9006dde0 msgr2=0x7f8d900cb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:24.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.370+0000 7f8d96a00640 1 --2- 172.21.15.138:0/2454588535 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8d9006dde0 0x7f8d900cb830 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8d800097d0 tx=0x7f8d8002b9c0 comp rx=0 tx=0).stop 2024-01-21T22:53:24.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.371+0000 7f8d96a00640 1 -- 172.21.15.138:0/2454588535 shutdown_connections 2024-01-21T22:53:24.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.371+0000 7f8d96a00640 1 --2- 172.21.15.138:0/2454588535 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8d9006dde0 0x7f8d900cb830 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:24.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.371+0000 7f8d96a00640 1 -- 172.21.15.138:0/2454588535 >> 172.21.15.138:0/2454588535 conn(0x7f8d9006add0 msgr2=0x7f8d900caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:53:24.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.371+0000 7f8d96a00640 1 -- 172.21.15.138:0/2454588535 shutdown_connections 2024-01-21T22:53:24.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.371+0000 7f8d96a00640 1 -- 172.21.15.138:0/2454588535 wait complete. 2024-01-21T22:53:24.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.373+0000 7f8d96a00640 1 Processor -- start 2024-01-21T22:53:24.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.373+0000 7f8d96a00640 1 -- start start 2024-01-21T22:53:24.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.374+0000 7f8d96a00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8d9006dde0 0x7f8d9015f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:24.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.374+0000 7f8d96a00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d9006e1c0 0x7f8d9015d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:24.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.374+0000 7f8d96a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d90157d10 0x7f8d901568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:24.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.374+0000 7f8d96a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9015f6a0 con 0x7f8d90157d10 2024-01-21T22:53:24.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.374+0000 7f8d96a00640 1 -- --> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d901603a0 con 0x7f8d9006e1c0 2024-01-21T22:53:24.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.374+0000 7f8d96a00640 1 -- --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d900ca530 con 0x7f8d9006dde0 2024-01-21T22:53:24.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.374+0000 7f8d87e00640 1 --2- >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d9006e1c0 0x7f8d9015d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:24.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.374+0000 7f8d94a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d90157d10 0x7f8d901568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:24.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.374+0000 7f8d8fe00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8d9006dde0 0x7f8d9015f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:24.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.375+0000 7f8d8fe00640 1 --2- >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8d9006dde0 0x7f8d9015f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.152:3301/0 says I am v2:172.21.15.138:44362/0 (socket says 172.21.15.138:44362) 2024-01-21T22:53:24.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.375+0000 7f8d8fe00640 1 -- 172.21.15.138:0/1096998562 learned_addr learned my addr 172.21.15.138:0/1096998562 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-01-21T22:53:24.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.378+0000 7f8d8fe00640 1 -- 172.21.15.138:0/1096998562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d9006e1c0 msgr2=0x7f8d9015d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:24.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.378+0000 7f8d8fe00640 1 --2- 172.21.15.138:0/1096998562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d9006e1c0 0x7f8d9015d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:24.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.378+0000 7f8d8fe00640 1 -- 172.21.15.138:0/1096998562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d90157d10 msgr2=0x7f8d901568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:24.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.378+0000 7f8d8fe00640 1 --2- 172.21.15.138:0/1096998562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d90157d10 0x7f8d901568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:24.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.378+0000 7f8d8fe00640 1 -- 172.21.15.138:0/1096998562 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d80009400 con 0x7f8d9006dde0 2024-01-21T22:53:24.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.380+0000 7f8d8fe00640 1 --2- 172.21.15.138:0/1096998562 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8d9006dde0 0x7f8d9015f160 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8d800098f0 tx=0x7f8d80004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:24.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.383+0000 7f8d8cc00640 1 -- 172.21.15.138:0/1096998562 <== mon.2 v2:172.21.15.152:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8003e040 con 0x7f8d9006dde0 2024-01-21T22:53:24.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.384+0000 7f8d8cc00640 1 -- 172.21.15.138:0/1096998562 <== mon.2 v2:172.21.15.152:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f8d80039d80 con 0x7f8d9006dde0 2024-01-21T22:53:24.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.384+0000 7f8d96a00640 1 -- 172.21.15.138:0/1096998562 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d90160e90 con 0x7f8d9006dde0 2024-01-21T22:53:24.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.384+0000 7f8d8cc00640 1 -- 172.21.15.138:0/1096998562 <== mon.2 v2:172.21.15.152:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d800436a0 con 0x7f8d9006dde0 2024-01-21T22:53:24.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.384+0000 7f8d96a00640 1 -- 172.21.15.138:0/1096998562 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d901557d0 con 0x7f8d9006dde0 2024-01-21T22:53:24.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.385+0000 7f8d96a00640 1 -- 172.21.15.138:0/1096998562 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d90161d90 con 0x7f8d9006dde0 2024-01-21T22:53:24.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.634+0000 7f8d8cc00640 1 -- 172.21.15.138:0/1096998562 <== mon.2 v2:172.21.15.152:3301/0 4 ==== mgrmap(e 5) v1 ==== 89516+0+0 (secure 0 0 0) 0x7f8d800423e0 con 0x7f8d9006dde0 2024-01-21T22:53:24.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.635+0000 7f8d8cc00640 1 --2- 172.21.15.138:0/1096998562 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f8d6806b380 0x7f8d6806d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-01-21T22:53:24.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.636+0000 7f8d8cc00640 1 -- 172.21.15.138:0/1096998562 <== mon.2 v2:172.21.15.152:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5386+0+0 (secure 0 0 0) 0x7f8d800ac590 con 0x7f8d9006dde0 2024-01-21T22:53:24.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.636+0000 7f8d87e00640 1 --2- 172.21.15.138:0/1096998562 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f8d6806b380 0x7f8d6806d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-01-21T22:53:24.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.637+0000 7f8d87e00640 1 --2- 172.21.15.138:0/1096998562 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f8d6806b380 0x7f8d6806d840 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8d9006cdd0 tx=0x7f8d78009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-01-21T22:53:24.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.647+0000 7f8d8cc00640 1 -- 172.21.15.138:0/1096998562 <== mon.2 v2:172.21.15.152:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180410 (secure 0 0 0) 0x7f8d800774f0 con 0x7f8d9006dde0 2024-01-21T22:53:24.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:24.867+0000 7f8d96a00640 1 -- 172.21.15.138:0/1096998562 --> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8d90161d90 con 0x7f8d9006dde0 2024-01-21T22:53:25.528 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-01-21T22:53:25.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:25.823+0000 7f8d8cc00640 1 -- 172.21.15.138:0/1096998562 <== mon.2 v2:172.21.15.152:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v28) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8d90161d90 con 0x7f8d9006dde0 2024-01-21T22:53:25.825 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_data' removed 2024-01-21T22:53:25.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:25.827+0000 7f8d96a00640 1 -- 172.21.15.138:0/1096998562 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f8d6806b380 msgr2=0x7f8d6806d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:25.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:25.827+0000 7f8d96a00640 1 --2- 172.21.15.138:0/1096998562 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f8d6806b380 0x7f8d6806d840 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8d9006cdd0 tx=0x7f8d78009040 comp rx=0 tx=0).stop 2024-01-21T22:53:25.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:25.827+0000 7f8d96a00640 1 -- 172.21.15.138:0/1096998562 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8d9006dde0 msgr2=0x7f8d9015f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-01-21T22:53:25.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:25.827+0000 7f8d96a00640 1 --2- 172.21.15.138:0/1096998562 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8d9006dde0 0x7f8d9015f160 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8d800098f0 tx=0x7f8d80004730 comp rx=0 tx=0).stop 2024-01-21T22:53:25.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:25.828+0000 7f8d96a00640 1 -- 172.21.15.138:0/1096998562 shutdown_connections 2024-01-21T22:53:25.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:25.828+0000 7f8d96a00640 1 --2- 172.21.15.138:0/1096998562 >> [v2:172.21.15.152:6800/314625564,v1:172.21.15.152:6801/314625564] conn(0x7f8d6806b380 0x7f8d6806d840 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:25.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:25.828+0000 7f8d96a00640 1 --2- 172.21.15.138:0/1096998562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d90157d10 0x7f8d901568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:25.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:25.828+0000 7f8d96a00640 1 --2- 172.21.15.138:0/1096998562 >> [v2:172.21.15.152:3300/0,v1:172.21.15.152:6789/0] conn(0x7f8d9006e1c0 0x7f8d9015d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:25.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:25.828+0000 7f8d96a00640 1 --2- 172.21.15.138:0/1096998562 >> [v2:172.21.15.152:3301/0,v1:172.21.15.152:6790/0] conn(0x7f8d9006dde0 0x7f8d9015f160 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-01-21T22:53:25.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:25.828+0000 7f8d96a00640 1 -- 172.21.15.138:0/1096998562 >> 172.21.15.138:0/1096998562 conn(0x7f8d9006add0 msgr2=0x7f8d9015e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-01-21T22:53:25.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:25.828+0000 7f8d96a00640 1 -- 172.21.15.138:0/1096998562 shutdown_connections 2024-01-21T22:53:25.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-01-21T22:53:25.828+0000 7f8d96a00640 1 -- 172.21.15.138:0/1096998562 wait complete. 2024-01-21T22:53:25.849 INFO:teuthology.misc:Shutting down mds daemons... 2024-01-21T22:53:25.850 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-01-21T22:53:25.850 INFO:teuthology.orchestra.run:waiting for 300 2024-01-21T22:53:25.867 INFO:tasks.ceph.mds.a.smithi138.stderr:2024-01-21T22:53:25.854+0000 b5cf640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i a (PID: 40639) UID: 0 2024-01-21T22:53:25.868 INFO:tasks.ceph.mds.a.smithi138.stderr:2024-01-21T22:53:25.855+0000 b5cf640 -1 mds.a *** got signal Terminated *** 2024-01-21T22:53:31.951 INFO:tasks.ceph.mds.a:Stopped 2024-01-21T22:53:31.952 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-01-21T22:53:31.952 INFO:teuthology.orchestra.run:waiting for 300 2024-01-21T22:53:31.993 INFO:tasks.ceph.mds.c.smithi138.stderr:2024-01-21T22:53:31.961+0000 b5cf640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i c (PID: 40641) UID: 0 2024-01-21T22:53:31.994 INFO:tasks.ceph.mds.c.smithi138.stderr:2024-01-21T22:53:31.966+0000 b5cf640 -1 mds.c *** got signal Terminated *** 2024-01-21T22:53:32.245 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-01-21T22:53:32.245 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~7s 2024-01-21T22:53:38.053 INFO:tasks.ceph.mds.c:Stopped 2024-01-21T22:53:38.054 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-01-21T22:53:38.054 INFO:teuthology.orchestra.run:waiting for 300 2024-01-21T22:53:38.095 INFO:tasks.ceph.mds.e.smithi138.stderr:2024-01-21T22:53:38.064+0000 b5cf640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.e.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i e (PID: 40644) UID: 0 2024-01-21T22:53:38.096 INFO:tasks.ceph.mds.e.smithi138.stderr:2024-01-21T22:53:38.069+0000 b5cf640 -1 mds.e *** got signal Terminated *** 2024-01-21T22:53:38.858 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~14s 2024-01-21T22:53:38.858 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-01-21T22:53:43.403 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:53:43.412 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:53:44.155 INFO:tasks.ceph.mds.e:Stopped 2024-01-21T22:53:44.156 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-01-21T22:53:44.156 INFO:teuthology.orchestra.run:waiting for 300 2024-01-21T22:53:44.168 INFO:tasks.ceph.mds.b.smithi152.stderr:2024-01-21T22:53:44.160+0000 b5cf640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i b (PID: 43344) UID: 0 2024-01-21T22:53:44.169 INFO:tasks.ceph.mds.b.smithi152.stderr:2024-01-21T22:53:44.161+0000 b5cf640 -1 mds.b *** got signal Terminated *** 2024-01-21T22:53:45.372 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~20s 2024-01-21T22:53:50.260 INFO:tasks.ceph.mds.b:Stopped 2024-01-21T22:53:50.260 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-01-21T22:53:50.261 INFO:teuthology.orchestra.run:waiting for 300 2024-01-21T22:53:50.261 DEBUG:teuthology.orchestra.run:got remote process result: 42 2024-01-21T22:53:50.261 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi152 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.d.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i d" 2024-01-21T22:53:50.262 INFO:tasks.ceph.mds.d:Stopped 2024-01-21T22:53:50.262 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-01-21T22:53:50.262 INFO:teuthology.orchestra.run:waiting for 300 2024-01-21T22:53:50.276 INFO:tasks.ceph.mds.f.smithi152.stderr:2024-01-21T22:53:50.266+0000 b5cf640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mds.f.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph-mds -f --cluster ceph -i f (PID: 43348) UID: 0 2024-01-21T22:53:50.276 INFO:tasks.ceph.mds.f.smithi152.stderr:2024-01-21T22:53:50.267+0000 b5cf640 -1 mds.f *** got signal Terminated *** 2024-01-21T22:53:51.783 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-01-21T22:53:56.364 INFO:tasks.ceph.mds.f:Stopped 2024-01-21T22:53:56.364 INFO:teuthology.misc:Shutting down osd daemons... 2024-01-21T22:53:56.365 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-01-21T22:53:56.365 INFO:teuthology.orchestra.run:waiting for 300 2024-01-21T22:53:56.366 INFO:tasks.ceph.osd.0.smithi138.stderr:2024-01-21T22:53:56.366+0000 7f7a2c600640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 0 (PID: 35887) UID: 0 2024-01-21T22:53:56.367 INFO:tasks.ceph.osd.0.smithi138.stderr:2024-01-21T22:53:56.366+0000 7f7a2c600640 -1 osd.0 33 *** Got signal Terminated *** 2024-01-21T22:54:02.467 INFO:tasks.ceph.osd.0:Stopped 2024-01-21T22:54:02.468 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-01-21T22:54:02.468 INFO:teuthology.orchestra.run:waiting for 300 2024-01-21T22:54:02.469 INFO:tasks.ceph.osd.1.smithi138.stderr:2024-01-21T22:54:02.469+0000 7fe1c3200640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 1 (PID: 35886) UID: 0 2024-01-21T22:54:02.470 INFO:tasks.ceph.osd.1.smithi138.stderr:2024-01-21T22:54:02.469+0000 7fe1c3200640 -1 osd.1 34 *** Got signal Terminated *** 2024-01-21T22:54:08.569 INFO:tasks.ceph.osd.1:Stopped 2024-01-21T22:54:08.569 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-01-21T22:54:08.570 INFO:teuthology.orchestra.run:waiting for 300 2024-01-21T22:54:08.571 INFO:tasks.ceph.osd.2.smithi138.stderr:2024-01-21T22:54:08.570+0000 7f3a58c00640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 2 (PID: 35890) UID: 0 2024-01-21T22:54:08.571 INFO:tasks.ceph.osd.2.smithi138.stderr:2024-01-21T22:54:08.570+0000 7f3a58c00640 -1 osd.2 36 *** Got signal Terminated *** 2024-01-21T22:54:13.534 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:54:13.543 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:54:14.672 INFO:tasks.ceph.osd.2:Stopped 2024-01-21T22:54:14.672 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-01-21T22:54:14.672 INFO:teuthology.orchestra.run:waiting for 300 2024-01-21T22:54:14.674 INFO:tasks.ceph.osd.3.smithi138.stderr:2024-01-21T22:54:14.672+0000 7f5d7d600640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 3 (PID: 35893) UID: 0 2024-01-21T22:54:14.674 INFO:tasks.ceph.osd.3.smithi138.stderr:2024-01-21T22:54:14.672+0000 7f5d7d600640 -1 osd.3 37 *** Got signal Terminated *** 2024-01-21T22:54:20.775 INFO:tasks.ceph.osd.3:Stopped 2024-01-21T22:54:20.775 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-01-21T22:54:20.775 INFO:teuthology.orchestra.run:waiting for 300 2024-01-21T22:54:20.776 INFO:tasks.ceph.osd.4.smithi152.stderr:2024-01-21T22:54:20.775+0000 7f286a2dc640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 4 (PID: 37156) UID: 0 2024-01-21T22:54:20.776 INFO:tasks.ceph.osd.4.smithi152.stderr:2024-01-21T22:54:20.775+0000 7f286a2dc640 -1 osd.4 38 *** Got signal Terminated *** 2024-01-21T22:54:26.341 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:30:10.534 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:54:26.358 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:30:10.551 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:54:26.359 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:30:10.617 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:54:26.361 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:30:10.619 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:54:26.363 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:30:10.621 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:54:26.367 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:30:10.625 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:54:26.476 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:30:10.734 36486== Warning: unimplemented fcntl command: 1036 2024-01-21T22:54:26.482 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:30:10.740 36485== Warning: unimplemented fcntl command: 1036 2024-01-21T22:54:26.506 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:30:10.699 35603== Warning: unimplemented fcntl command: 1036 2024-01-21T22:54:26.877 INFO:tasks.ceph.osd.4:Stopped 2024-01-21T22:54:26.877 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-01-21T22:54:26.877 INFO:teuthology.orchestra.run:waiting for 300 2024-01-21T22:54:26.878 INFO:tasks.ceph.osd.5.smithi152.stderr:2024-01-21T22:54:26.877+0000 7f4b1a49d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 5 (PID: 37159) UID: 0 2024-01-21T22:54:26.878 INFO:tasks.ceph.osd.5.smithi152.stderr:2024-01-21T22:54:26.877+0000 7f4b1a49d640 -1 osd.5 39 *** Got signal Terminated *** 2024-01-21T22:54:32.979 INFO:tasks.ceph.osd.5:Stopped 2024-01-21T22:54:32.979 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-01-21T22:54:32.979 INFO:teuthology.orchestra.run:waiting for 300 2024-01-21T22:54:32.980 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-01-21T22:54:32.979+0000 7f0780b13640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 6 (PID: 37158) UID: 0 2024-01-21T22:54:32.980 INFO:tasks.ceph.osd.6.smithi152.stderr:2024-01-21T22:54:32.979+0000 7f0780b13640 -1 osd.6 40 *** Got signal Terminated *** 2024-01-21T22:54:39.080 INFO:tasks.ceph.osd.6:Stopped 2024-01-21T22:54:39.081 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-01-21T22:54:39.081 INFO:teuthology.orchestra.run:waiting for 300 2024-01-21T22:54:39.082 INFO:tasks.ceph.osd.7.smithi152.stderr:2024-01-21T22:54:39.081+0000 7fea2cbdb640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-osd -f --cluster ceph -i 7 (PID: 37162) UID: 0 2024-01-21T22:54:39.082 INFO:tasks.ceph.osd.7.smithi152.stderr:2024-01-21T22:54:39.081+0000 7fea2cbdb640 -1 osd.7 40 *** Got signal Terminated *** 2024-01-21T22:54:39.449 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-01-21T22:54:43.684 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:54:43.693 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:54:44.953 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-01-21T22:54:44.953 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-01-21T22:54:45.182 INFO:tasks.ceph.osd.7:Stopped 2024-01-21T22:54:45.182 INFO:teuthology.misc:Shutting down mgr daemons... 2024-01-21T22:54:45.183 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-01-21T22:54:45.183 INFO:teuthology.orchestra.run:waiting for 300 2024-01-21T22:54:45.260 INFO:tasks.ceph.mgr.y:Stopped 2024-01-21T22:54:45.260 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-01-21T22:54:45.260 INFO:teuthology.orchestra.run:waiting for 300 2024-01-21T22:54:45.261 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:54:45.260+0000 7fc100583640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term ceph-mgr -f --cluster ceph -i x (PID: 36483) UID: 0 2024-01-21T22:54:45.261 INFO:tasks.ceph.mgr.x.smithi152.stderr:2024-01-21T22:54:45.260+0000 7fc100583640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-01-21T22:54:45.287 INFO:tasks.ceph.mgr.x:Stopped 2024-01-21T22:54:45.287 INFO:teuthology.misc:Shutting down mon daemons... 2024-01-21T22:54:45.287 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-01-21T22:54:45.287 INFO:teuthology.orchestra.run:waiting for 300 2024-01-21T22:54:45.349 INFO:tasks.ceph.mon.a.smithi138.stderr:2024-01-21T22:54:45.316+0000 9f79640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a (PID: 35591) UID: 0 2024-01-21T22:54:45.350 INFO:tasks.ceph.mon.a.smithi138.stderr:2024-01-21T22:54:45.320+0000 9f79640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-01-21T22:54:46.278 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:30:30.472 35603== 2024-01-21T22:54:46.279 INFO:tasks.ceph.mon.a.smithi138.stderr:==00:00:30:30.472 35603== Exit program on first error (--exit-on-first-error=yes) 2024-01-21T22:54:46.327 INFO:tasks.ceph.mon.a.smithi138.stderr:daemon-helper: command failed with exit status 42 2024-01-21T22:54:50.156 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-01-21T22:54:50.156 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-01-21T22:54:51.388 DEBUG:teuthology.orchestra.run:got remote process result: 42 2024-01-21T22:54:51.388 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi138 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a" 2024-01-21T22:54:51.389 INFO:tasks.ceph.mon.a:Stopped 2024-01-21T22:54:51.389 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-01-21T22:54:51.389 INFO:teuthology.orchestra.run:waiting for 300 2024-01-21T22:54:51.405 INFO:tasks.ceph.mon.b.smithi152.stderr:2024-01-21T22:54:51.392+0000 9f79640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b (PID: 36478) UID: 0 2024-01-21T22:54:51.405 INFO:tasks.ceph.mon.b.smithi152.stderr:2024-01-21T22:54:51.393+0000 9f79640 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2024-01-21T22:54:52.053 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:30:36.311 36485== 2024-01-21T22:54:52.053 INFO:tasks.ceph.mon.b.smithi152.stderr:==00:00:30:36.311 36485== Exit program on first error (--exit-on-first-error=yes) 2024-01-21T22:54:52.063 INFO:tasks.ceph.mon.b.smithi152.stderr:daemon-helper: command failed with exit status 42 2024-01-21T22:54:55.258 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-01-21T22:54:55.258 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-01-21T22:54:57.490 DEBUG:teuthology.orchestra.run:got remote process result: 42 2024-01-21T22:54:57.491 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi152 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b" 2024-01-21T22:54:57.491 INFO:tasks.ceph.mon.b:Stopped 2024-01-21T22:54:57.491 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-01-21T22:54:57.492 INFO:teuthology.orchestra.run:waiting for 300 2024-01-21T22:54:57.525 INFO:tasks.ceph.mon.c.smithi152.stderr:2024-01-21T22:54:57.494+0000 9f79640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term env OPENSSL_ia32cap=~0x1000000000000000 valgrind --trace-children=no --child-silent-after-fork=yes --soname-synonyms=somalloc=*tcmalloc* --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c (PID: 36480) UID: 0 2024-01-21T22:54:57.525 INFO:tasks.ceph.mon.c.smithi152.stderr:2024-01-21T22:54:57.496+0000 9f79640 -1 mon.c@2(probing) e1 *** Got Signal Terminated *** 2024-01-21T22:54:58.149 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:30:42.407 36486== 2024-01-21T22:54:58.149 INFO:tasks.ceph.mon.c.smithi152.stderr:==00:00:30:42.407 36486== Exit program on first error (--exit-on-first-error=yes) 2024-01-21T22:54:58.160 INFO:tasks.ceph.mon.c.smithi152.stderr:daemon-helper: command failed with exit status 42 2024-01-21T22:55:00.259 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-01-21T22:55:00.259 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-01-21T22:55:03.594 DEBUG:teuthology.orchestra.run:got remote process result: 42 2024-01-21T22:55:03.594 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi152 with status 42: "cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c" 2024-01-21T22:55:03.594 INFO:tasks.ceph.mon.c:Stopped 2024-01-21T22:55:03.594 INFO:tasks.ceph:Checking cluster log for badness... 2024-01-21T22:55:03.595 DEBUG:teuthology.orchestra.run.smithi138:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow request' | egrep -v SLOW_OPS | egrep -v MON_DOWN | head -n 1 2024-01-21T22:55:03.701 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi138.front.sepia.ceph.com 2024-01-21T22:55:03.701 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-01-21T22:55:03.860 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi138.front.sepia.ceph.com 2024-01-21T22:55:03.860 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-01-21T22:55:04.029 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi138.front.sepia.ceph.com 2024-01-21T22:55:04.029 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-01-21T22:55:04.201 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi138.front.sepia.ceph.com 2024-01-21T22:55:04.201 DEBUG:teuthology.orchestra.run.smithi138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-01-21T22:55:04.417 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi152.front.sepia.ceph.com 2024-01-21T22:55:04.417 DEBUG:teuthology.orchestra.run.smithi152:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-01-21T22:55:04.492 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi152.front.sepia.ceph.com 2024-01-21T22:55:04.493 DEBUG:teuthology.orchestra.run.smithi152:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-01-21T22:55:04.571 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi152.front.sepia.ceph.com 2024-01-21T22:55:04.572 DEBUG:teuthology.orchestra.run.smithi152:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-01-21T22:55:04.635 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi152.front.sepia.ceph.com 2024-01-21T22:55:04.636 DEBUG:teuthology.orchestra.run.smithi152:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-01-21T22:55:04.718 INFO:tasks.ceph:Archiving mon data... 2024-01-21T22:55:04.719 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2024-01-19_10:13:34-fs-wip-vshankar-testing-20240119.075157-testing-default-smithi/7522024/data/mon.a.tgz 2024-01-21T22:55:04.719 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-01-21T22:55:04.776 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.y8hdt36kdE 2024-01-21T22:55:04.777 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.y8hdt36kdE 2024-01-21T22:55:04.938 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.y8hdt36kdE 2024-01-21T22:55:05.090 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.y8hdt36kdE is 273KB 2024-01-21T22:55:05.199 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.y8hdt36kdE 2024-01-21T22:55:05.256 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2024-01-19_10:13:34-fs-wip-vshankar-testing-20240119.075157-testing-default-smithi/7522024/data/mon.b.tgz 2024-01-21T22:55:05.256 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2024-01-21T22:55:05.260 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-01-21T22:55:05.274 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.gu6XsnsiqL 2024-01-21T22:55:05.274 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.gu6XsnsiqL 2024-01-21T22:55:05.405 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.gu6XsnsiqL 2024-01-21T22:55:05.492 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.gu6XsnsiqL is 274KB 2024-01-21T22:55:05.587 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.gu6XsnsiqL 2024-01-21T22:55:05.602 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2024-01-19_10:13:34-fs-wip-vshankar-testing-20240119.075157-testing-default-smithi/7522024/data/mon.c.tgz 2024-01-21T22:55:05.603 DEBUG:teuthology.orchestra.run.smithi152:> mktemp 2024-01-21T22:55:05.657 INFO:teuthology.orchestra.run.smithi152.stdout:/tmp/tmp.6q8ygolVq5 2024-01-21T22:55:05.657 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.6q8ygolVq5 2024-01-21T22:55:05.771 DEBUG:teuthology.orchestra.run.smithi152:> sudo chmod 0666 /tmp/tmp.6q8ygolVq5 2024-01-21T22:55:05.857 DEBUG:teuthology.orchestra.remote:smithi152:/tmp/tmp.6q8ygolVq5 is 274KB 2024-01-21T22:55:05.922 DEBUG:teuthology.orchestra.run.smithi152:> rm -fr /tmp/tmp.6q8ygolVq5 2024-01-21T22:55:05.936 INFO:tasks.ceph:Cleaning ceph cluster... 2024-01-21T22:55:05.936 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-01-21T22:55:05.944 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-01-21T22:55:06.322 DEBUG:tasks.ceph:valgrind exception message: valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun 2024-01-21T22:55:06.559 INFO:tasks.ceph:Archiving crash dumps... 2024-01-21T22:55:06.561 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-01-19_10:13:34-fs-wip-vshankar-testing-20240119.075157-testing-default-smithi/7522024/remote/smithi138/crash 2024-01-21T22:55:06.562 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-21T22:55:06.659 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-01-19_10:13:34-fs-wip-vshankar-testing-20240119.075157-testing-default-smithi/7522024/remote/smithi152/crash 2024-01-21T22:55:06.660 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-01-21T22:55:06.686 INFO:tasks.ceph:Shutting down logrotate 2024-01-21T22:55:06.686 DEBUG:teuthology.orchestra.run.smithi138:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:55:06.694 DEBUG:teuthology.orchestra.run.smithi152:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-01-21T22:55:06.820 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-01-21T22:55:06.918 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-01-21T22:55:06.945 INFO:tasks.ceph:Compressing logs... 2024-01-21T22:55:06.945 DEBUG:teuthology.orchestra.run.smithi138:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-21T22:55:06.953 DEBUG:teuthology.orchestra.run.smithi152:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-01-21T22:55:07.012 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-01-21T22:55:07.012 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37064.log 2024-01-21T22:55:07.020 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36797.log 2024-01-21T22:55:07.020 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.37064.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.c.log 2024-01-21T22:55:07.021 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.36797.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37064.log.gz 2024-01-21T22:55:07.021 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/mds.f.log 2024-01-21T22:55:07.021 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/mds.b.log 2024-01-21T22:55:07.021 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/valgrind/mon.c.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36797.log.gz 2024-01-21T22:55:07.021 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/valgrind/mds.f.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.b.log 2024-01-21T22:55:07.021 INFO:teuthology.orchestra.run.smithi152.stderr: 69.6% -- replaced with /var/log/ceph/valgrind/mon.c.log.gz 2024-01-21T22:55:07.021 INFO:teuthology.orchestra.run.smithi152.stderr: 57.2% -- replaced with /var/log/ceph/valgrind/mds.f.log.gz 2024-01-21T22:55:07.021 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/valgrind/mds.b.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mds.d.log 2024-01-21T22:55:07.021 INFO:teuthology.orchestra.run.smithi152.stderr: 57.2% -- replaced with /var/log/ceph/valgrind/mds.b.log.gz 2024-01-21T22:55:07.021 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/valgrind/mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36677.log 2024-01-21T22:55:07.021 INFO:teuthology.orchestra.run.smithi152.stderr: 69.6% -- replaced with /var/log/ceph/valgrind/mon.b.log.gz 2024-01-21T22:55:07.021 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/valgrind/mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36906.log 2024-01-21T22:55:07.021 INFO:teuthology.orchestra.run.smithi152.stderr: 67.7% -- replaced with /var/log/ceph/valgrind/mds.d.log.gz 2024-01-21T22:55:07.021 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.36677.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-01-21T22:55:07.022 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.36906.log: gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.36677.log.gz --verbose 2024-01-21T22:55:07.022 INFO:teuthology.orchestra.run.smithi152.stderr: -- /var/log/ceph/ceph-mds.b.log 2024-01-21T22:55:07.022 INFO:teuthology.orchestra.run.smithi152.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36906.log.gz 2024-01-21T22:55:07.022 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-01-21T22:55:07.022 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36950.log 2024-01-21T22:55:07.022 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-01-21T22:55:07.022 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.36950.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-01-21T22:55:07.022 INFO:teuthology.orchestra.run.smithi152.stderr: 86.4%/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-01-21T22:55:07.022 INFO:teuthology.orchestra.run.smithi152.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36950.log.gz 2024-01-21T22:55:07.022 INFO:teuthology.orchestra.run.smithi152.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-01-21T22:55:07.022 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-01-21T22:55:07.022 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36848.log/var/log/ceph/ceph-mds.f.log: 2024-01-21T22:55:07.022 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-01-21T22:55:07.022 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.36848.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36848.log.gz 2024-01-21T22:55:07.023 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-01-21T22:55:07.026 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph.log: 92.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-01-21T22:55:07.026 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36717.log 2024-01-21T22:55:07.027 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-mon.b.log: 92.6% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-01-21T22:55:07.027 INFO:teuthology.orchestra.run.smithi152.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-01-21T22:55:07.027 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.36717.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36717.log.gz 2024-01-21T22:55:07.027 INFO:teuthology.orchestra.run.smithi152.stderr: 93.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36757.log 2024-01-21T22:55:07.027 INFO:teuthology.orchestra.run.smithi152.stderr: -- replaced with /var/log/ceph/ceph.log.gz 2024-01-21T22:55:07.027 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-01-21T22:55:07.027 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-client.admin.36757.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36757.log.gz 2024-01-21T22:55:07.036 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41869.log 2024-01-21T22:55:07.037 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41351.log 2024-01-21T22:55:07.038 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42457.log 2024-01-21T22:55:07.038 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41869.log.gz 2024-01-21T22:55:07.039 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40694.log 2024-01-21T22:55:07.039 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41351.log.gz 2024-01-21T22:55:07.040 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42457.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-01-21T22:55:07.040 INFO:teuthology.orchestra.run.smithi138.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42457.log.gz 2024-01-21T22:55:07.041 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.40694.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40694.log.gz 2024-01-21T22:55:07.042 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41165.log 2024-01-21T22:55:07.043 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42198.log 2024-01-21T22:55:07.044 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41861.log 2024-01-21T22:55:07.044 INFO:teuthology.orchestra.run.smithi138.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41165.log.gz 2024-01-21T22:55:07.045 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45618.log 2024-01-21T22:55:07.045 INFO:teuthology.orchestra.run.smithi138.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42198.log.gz 2024-01-21T22:55:07.046 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42401.log 2024-01-21T22:55:07.046 INFO:teuthology.orchestra.run.smithi138.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41861.log.gz 2024-01-21T22:55:07.047 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.45618.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45618.log.gz -5 2024-01-21T22:55:07.047 INFO:teuthology.orchestra.run.smithi138.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40955.log 2024-01-21T22:55:07.049 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46897.log 2024-01-21T22:55:07.049 INFO:teuthology.orchestra.run.smithi138.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42401.log.gz 2024-01-21T22:55:07.050 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.40955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42160.log 2024-01-21T22:55:07.050 INFO:teuthology.orchestra.run.smithi138.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40955.log.gz 2024-01-21T22:55:07.051 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46897.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mds.c.log 2024-01-21T22:55:07.051 INFO:teuthology.orchestra.run.smithi138.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46897.log.gz 2024-01-21T22:55:07.052 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42160.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42160.log.gz 2024-01-21T22:55:07.052 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/mds.a.log 2024-01-21T22:55:07.053 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/valgrind/mds.c.log: gzip -5 --verbose -- /var/log/ceph/valgrind/mon.a.log 2024-01-21T22:55:07.054 INFO:teuthology.orchestra.run.smithi138.stderr: 57.3% -- replaced with /var/log/ceph/valgrind/mds.c.log.gz 2024-01-21T22:55:07.054 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/valgrind/mds.a.log: gzip 57.2% -- replaced with /var/log/ceph/valgrind/mds.a.log.gz 2024-01-21T22:55:07.055 INFO:teuthology.orchestra.run.smithi138.stderr: -5 --verbose -- /var/log/ceph/valgrind/mds.e.log 2024-01-21T22:55:07.056 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/valgrind/mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42619.log 2024-01-21T22:55:07.056 INFO:teuthology.orchestra.run.smithi138.stderr: 69.6% -- replaced with /var/log/ceph/valgrind/mon.a.log.gz 2024-01-21T22:55:07.057 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/valgrind/mds.e.log: 57.3% -- replaced with /var/log/ceph/valgrind/mds.e.log.gz 2024-01-21T22:55:07.057 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42737.log 2024-01-21T22:55:07.058 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41225.log 2024-01-21T22:55:07.058 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42619.log.gz 2024-01-21T22:55:07.059 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46713.log 2024-01-21T22:55:07.059 INFO:teuthology.orchestra.run.smithi138.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42737.log.gz 2024-01-21T22:55:07.060 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35897.log 2024-01-21T22:55:07.061 INFO:teuthology.orchestra.run.smithi138.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41225.log.gz 2024-01-21T22:55:07.062 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46209.log 2024-01-21T22:55:07.062 INFO:teuthology.orchestra.run.smithi138.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46713.log.gz 2024-01-21T22:55:07.063 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.35897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42500.log 2024-01-21T22:55:07.063 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35897.log.gz 2024-01-21T22:55:07.063 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46209.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46209.log.gz 2024-01-21T22:55:07.064 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46083.log 2024-01-21T22:55:07.065 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35601.log 2024-01-21T22:55:07.065 INFO:teuthology.orchestra.run.smithi138.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42500.log.gz 2024-01-21T22:55:07.066 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46773.log 2024-01-21T22:55:07.066 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46083.log.gz 2024-01-21T22:55:07.067 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.35601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47024.log 2024-01-21T22:55:07.067 INFO:teuthology.orchestra.run.smithi138.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.35601.log.gz 2024-01-21T22:55:07.068 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46939.log 2024-01-21T22:55:07.068 INFO:teuthology.orchestra.run.smithi138.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46773.log.gz 2024-01-21T22:55:07.069 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46419.log 2024-01-21T22:55:07.069 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47024.log.gz 2024-01-21T22:55:07.070 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42874.log 2024-01-21T22:55:07.071 INFO:teuthology.orchestra.run.smithi138.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46939.log.gz 2024-01-21T22:55:07.072 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42482.log 2024-01-21T22:55:07.072 INFO:teuthology.orchestra.run.smithi138.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46419.log.gz 2024-01-21T22:55:07.073 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41519.log 2024-01-21T22:55:07.073 INFO:teuthology.orchestra.run.smithi138.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42874.log.gz 2024-01-21T22:55:07.074 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40736.log 2024-01-21T22:55:07.074 INFO:teuthology.orchestra.run.smithi138.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42482.log.gz 2024-01-21T22:55:07.075 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43493.log 2024-01-21T22:55:07.075 INFO:teuthology.orchestra.run.smithi138.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41519.log.gz 2024-01-21T22:55:07.076 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.40736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43283.log 2024-01-21T22:55:07.076 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40736.log.gz 2024-01-21T22:55:07.077 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43493.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43493.log.gz 2024-01-21T22:55:07.077 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41862.log 2024-01-21T22:55:07.078 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43283.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.29530.log 2024-01-21T22:55:07.079 INFO:teuthology.orchestra.run.smithi138.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43283.log.gz 2024-01-21T22:55:07.079 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42201.log 2024-01-21T22:55:07.079 INFO:teuthology.orchestra.run.smithi138.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41862.log.gz 2024-01-21T22:55:07.080 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph.tmp-client.admin.29530.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.29530.log.gz 2024-01-21T22:55:07.080 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43031.log 2024-01-21T22:55:07.081 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45576.log 2024-01-21T22:55:07.081 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42201.log.gz 2024-01-21T22:55:07.083 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43031.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43031.log.gz 2024-01-21T22:55:07.083 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43637.log 2024-01-21T22:55:07.083 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.45576.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-01-21T22:55:07.084 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45576.log.gz 2024-01-21T22:55:07.084 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43637.log: gzip -5 --verbose -- 88.3% /var/log/ceph/ceph-client.admin.46167.log -- replaced with /var/log/ceph/ceph-client.admin.43637.log.gz 2024-01-21T22:55:07.084 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-01-21T22:55:07.085 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46335.log 2024-01-21T22:55:07.087 INFO:teuthology.orchestra.run.smithi138.stderr: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-01-21T22:55:07.087 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42501.log 2024-01-21T22:55:07.087 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46167.log.gz 2024-01-21T22:55:07.087 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45660.log 2024-01-21T22:55:07.088 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46335.log.gz 2024-01-21T22:55:07.088 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42501.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42501.log.gz 2024-01-21T22:55:07.089 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41123.log 2024-01-21T22:55:07.090 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.45660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45534.log 2024-01-21T22:55:07.090 INFO:teuthology.orchestra.run.smithi138.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45660.log.gz 2024-01-21T22:55:07.091 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47066.log 2024-01-21T22:55:07.091 INFO:teuthology.orchestra.run.smithi138.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41123.log.gz 2024-01-21T22:55:07.092 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.45534.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45534.log.gz 2024-01-21T22:55:07.092 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41267.log 2024-01-21T22:55:07.093 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46293.log 2024-01-21T22:55:07.094 INFO:teuthology.orchestra.run.smithi138.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47066.log.gz 2024-01-21T22:55:07.094 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41267.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41267.log.gz 2024-01-21T22:55:07.094 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41866.log 2024-01-21T22:55:07.095 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46293.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46293.log.gz 2024-01-21T22:55:07.096 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42270.log 2024-01-21T22:55:07.097 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40834.log 2024-01-21T22:55:07.097 INFO:teuthology.orchestra.run.smithi138.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41866.log.gz 2024-01-21T22:55:07.098 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45945.log 88.0% 2024-01-21T22:55:07.098 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42270.log.gz 2024-01-21T22:55:07.099 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.40834.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-01-21T22:55:07.099 INFO:teuthology.orchestra.run.smithi138.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40834.log.gz 2024-01-21T22:55:07.101 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.45945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47108.log 2024-01-21T22:55:07.101 INFO:teuthology.orchestra.run.smithi138.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45945.log.gz 2024-01-21T22:55:07.101 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42183.log 2024-01-21T22:55:07.103 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.47108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42134.log 2024-01-21T22:55:07.103 INFO:teuthology.orchestra.run.smithi138.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47108.log.gz 2024-01-21T22:55:07.104 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46461.log 2024-01-21T22:55:07.104 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42183.log.gz 2024-01-21T22:55:07.106 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42134.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42134.log.gz 2024-01-21T22:55:07.106 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41868.log 2024-01-21T22:55:07.107 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45702.log 2024-01-21T22:55:07.107 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46461.log.gz 2024-01-21T22:55:07.108 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41561.log 88.8% 2024-01-21T22:55:07.108 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41868.log.gz 2024-01-21T22:55:07.110 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.45702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42150.log 2024-01-21T22:55:07.110 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45702.log.gz 2024-01-21T22:55:07.111 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41561.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41561.log.gz 2024-01-21T22:55:07.111 INFO:teuthology.orchestra.run.smithi138.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43241.log 2024-01-21T22:55:07.112 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42150.log: gzip 89.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.42150.log.gz --verbose 2024-01-21T22:55:07.112 INFO:teuthology.orchestra.run.smithi138.stderr: -- /var/log/ceph/ceph-client.admin.46545.log 2024-01-21T22:55:07.113 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46981.log 2024-01-21T22:55:07.114 INFO:teuthology.orchestra.run.smithi138.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43241.log.gz 2024-01-21T22:55:07.115 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46545.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46545.log.gz 2024-01-21T22:55:07.115 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43367.log 2024-01-21T22:55:07.116 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43115.log 2024-01-21T22:55:07.116 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46981.log.gz 2024-01-21T22:55:07.117 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43367.log: gzip -5 88.1% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.43367.log.gz 2024-01-21T22:55:07.117 INFO:teuthology.orchestra.run.smithi138.stderr: /var/log/ceph/ceph-client.admin.46503.log 2024-01-21T22:55:07.118 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46855.log 2024-01-21T22:55:07.118 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43115.log.gz 2024-01-21T22:55:07.119 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46503.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46503.log.gz 2024-01-21T22:55:07.120 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-01-21T22:55:07.121 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46587.log 2024-01-21T22:55:07.121 INFO:teuthology.orchestra.run.smithi138.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46855.log.gz 2024-01-21T22:55:07.122 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43595.log 2024-01-21T22:55:07.123 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46587.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46587.log.gz 2024-01-21T22:55:07.123 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42817.log 2024-01-21T22:55:07.125 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46125.log 2024-01-21T22:55:07.125 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43595.log.gz 2024-01-21T22:55:07.126 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43409.log 2024-01-21T22:55:07.127 INFO:teuthology.orchestra.run.smithi138.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42817.log.gz 2024-01-21T22:55:07.128 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46041.log 2024-01-21T22:55:07.128 INFO:teuthology.orchestra.run.smithi138.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46125.log.gz 2024-01-21T22:55:07.129 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40997.log 2024-01-21T22:55:07.129 INFO:teuthology.orchestra.run.smithi138.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43409.log.gz 2024-01-21T22:55:07.131 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41856.log 2024-01-21T22:55:07.131 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46041.log.gz 2024-01-21T22:55:07.132 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.40997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46671.log 2024-01-21T22:55:07.132 INFO:teuthology.orchestra.run.smithi138.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40997.log.gz 2024-01-21T22:55:07.133 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41645.log 2024-01-21T22:55:07.134 INFO:teuthology.orchestra.run.smithi138.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41856.log.gz 2024-01-21T22:55:07.135 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43325.log 2024-01-21T22:55:07.135 INFO:teuthology.orchestra.run.smithi138.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46671.log.gz 2024-01-21T22:55:07.136 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43199.log 2024-01-21T22:55:07.137 INFO:teuthology.orchestra.run.smithi138.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41645.log.gz 2024-01-21T22:55:07.138 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46251.log 2024-01-21T22:55:07.138 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43325.log.gz 2024-01-21T22:55:07.139 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43451.log 2024-01-21T22:55:07.140 INFO:teuthology.orchestra.run.smithi138.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43199.log.gz 2024-01-21T22:55:07.141 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40649.log 2024-01-21T22:55:07.141 INFO:teuthology.orchestra.run.smithi138.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46251.log.gz 2024-01-21T22:55:07.142 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43451.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-01-21T22:55:07.142 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43451.log.gz 2024-01-21T22:55:07.144 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.40649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41603.log 2024-01-21T22:55:07.144 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40649.log.gz 2024-01-21T22:55:07.145 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-01-21T22:55:07.147 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41477.log 2024-01-21T22:55:07.147 INFO:teuthology.orchestra.run.smithi138.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41603.log.gz 2024-01-21T22:55:07.148 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41865.log 2024-01-21T22:55:07.150 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44111.log 2024-01-21T22:55:07.150 INFO:teuthology.orchestra.run.smithi138.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41477.log.gz 2024-01-21T22:55:07.151 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43781.log 2024-01-21T22:55:07.151 INFO:teuthology.orchestra.run.smithi138.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41865.log.gz 2024-01-21T22:55:07.152 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.0.44111.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44111.log.gz 2024-01-21T22:55:07.152 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41081.log 2024-01-21T22:55:07.153 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43781.log: 92.4% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-01-21T22:55:07.154 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41435.log 2024-01-21T22:55:07.154 INFO:teuthology.orchestra.run.smithi138.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43781.log.gz 2024-01-21T22:55:07.155 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42117.log 2024-01-21T22:55:07.155 INFO:teuthology.orchestra.run.smithi138.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41081.log.gz 2024-01-21T22:55:07.156 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41435.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-01-21T22:55:07.157 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41435.log.gz 2024-01-21T22:55:07.158 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42117.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42117.log.gz 2024-01-21T22:55:07.158 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45996.log 2024-01-21T22:55:07.160 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46377.log 2024-01-21T22:55:07.162 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.45996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42947.log 2024-01-21T22:55:07.162 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45996.log.gz 2024-01-21T22:55:07.163 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46377.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46377.log.gz 2024-01-21T22:55:07.163 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41309.log 2024-01-21T22:55:07.165 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41687.log 2024-01-21T22:55:07.165 INFO:teuthology.orchestra.run.smithi138.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42947.log.gz 2024-01-21T22:55:07.166 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42678.log 2024-01-21T22:55:07.166 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41309.log.gz 2024-01-21T22:55:07.167 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41687.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41687.log.gz 2024-01-21T22:55:07.168 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40534.log 2024-01-21T22:55:07.169 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42678.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42678.log.gz 2024-01-21T22:55:07.169 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45843.log 2024-01-21T22:55:07.171 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.40534.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-01-21T22:55:07.171 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40534.log.gz 2024-01-21T22:55:07.172 INFO:teuthology.orchestra.run.smithi138.stderr: 94.9% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-01-21T22:55:07.172 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.45843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43679.log 2024-01-21T22:55:07.172 INFO:teuthology.orchestra.run.smithi138.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45843.log.gz 2024-01-21T22:55:07.173 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-01-21T22:55:07.174 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43679.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-01-21T22:55:07.175 INFO:teuthology.orchestra.run.smithi138.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.43679.log.gz 2024-01-21T22:55:07.176 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46629.log 2024-01-21T22:55:07.177 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mds.c.log: 87.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-01-21T22:55:07.178 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46629.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46629.log.gz 2024-01-21T22:55:07.179 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42877.log 2024-01-21T22:55:07.180 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42925.log 2024-01-21T22:55:07.182 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42877.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42877.log.gz 2024-01-21T22:55:07.183 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42925.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42925.log.gz 2024-01-21T22:55:07.184 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41393.log 2024-01-21T22:55:07.185 INFO:teuthology.orchestra.run.smithi138.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-01-21T22:55:07.185 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42593.log 2024-01-21T22:55:07.187 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43535.log 2024-01-21T22:55:07.187 INFO:teuthology.orchestra.run.smithi138.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41393.log.gz 2024-01-21T22:55:07.188 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36043.log 2024-01-21T22:55:07.188 INFO:teuthology.orchestra.run.smithi138.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42593.log.gz 2024-01-21T22:55:07.189 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45885.log 2024-01-21T22:55:07.190 INFO:teuthology.orchestra.run.smithi138.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43535.log.gz 2024-01-21T22:55:07.191 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.36043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41851.log 2024-01-21T22:55:07.191 INFO:teuthology.orchestra.run.smithi138.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36043.log.gz 2024-01-21T22:55:07.192 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.45885.log: 87.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46813.log 2024-01-21T22:55:07.192 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45885.log.gz 2024-01-21T22:55:07.194 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42767.log 2024-01-21T22:55:07.194 INFO:teuthology.orchestra.run.smithi138.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41851.log.gz 2024-01-21T22:55:07.195 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.46813.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46813.log.gz 2024-01-21T22:55:07.196 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43157.log 2024-01-21T22:55:07.196 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.42767.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42767.log.gz 2024-01-21T22:55:07.197 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-01-21T22:55:07.198 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41039.log 2024-01-21T22:55:07.199 INFO:teuthology.orchestra.run.smithi138.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43157.log.gz 2024-01-21T22:55:07.200 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40912.log 2024-01-21T22:55:07.201 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.41039.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41039.log.gz 2024-01-21T22:55:07.203 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.40912.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40912.log.gz 2024-01-21T22:55:07.203 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43073.log 2024-01-21T22:55:07.206 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.43073.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43073.log.gz 2024-01-21T22:55:07.207 INFO:teuthology.orchestra.run.smithi138.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-01-21T22:55:07.278 INFO:teuthology.orchestra.run.smithi152.stderr:/var/log/ceph/ceph-mon.c.log: 92.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-01-21T22:55:07.557 INFO:teuthology.orchestra.run.smithi152.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-01-21T22:55:07.686 INFO:teuthology.orchestra.run.smithi152.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-01-21T22:55:07.998 INFO:teuthology.orchestra.run.smithi138.stderr: 89.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-01-21T22:55:08.107 INFO:teuthology.orchestra.run.smithi152.stderr: 92.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-01-21T22:55:08.316 INFO:teuthology.orchestra.run.smithi138.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-01-21T22:55:08.375 INFO:teuthology.orchestra.run.smithi138.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-01-21T22:55:08.470 INFO:teuthology.orchestra.run.smithi152.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-01-21T22:55:08.794 INFO:teuthology.orchestra.run.smithi152.stderr: 92.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-01-21T22:55:08.952 INFO:teuthology.orchestra.run.smithi152.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-01-21T22:55:09.183 INFO:teuthology.orchestra.run.smithi138.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-01-21T22:55:09.320 INFO:teuthology.orchestra.run.smithi138.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-01-21T22:55:09.325 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-01-21T22:55:09.325 INFO:teuthology.orchestra.run.smithi138.stderr:real 0m2.328s 2024-01-21T22:55:09.325 INFO:teuthology.orchestra.run.smithi138.stderr:user 0m7.157s 2024-01-21T22:55:09.325 INFO:teuthology.orchestra.run.smithi138.stderr:sys 0m0.950s 2024-01-21T22:55:32.342 INFO:teuthology.orchestra.run.smithi152.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-01-21T22:55:32.343 INFO:teuthology.orchestra.run.smithi152.stderr: 2024-01-21T22:55:32.344 INFO:teuthology.orchestra.run.smithi152.stderr:real 0m25.345s 2024-01-21T22:55:32.344 INFO:teuthology.orchestra.run.smithi152.stderr:user 0m32.045s 2024-01-21T22:55:32.344 INFO:teuthology.orchestra.run.smithi152.stderr:sys 0m0.994s 2024-01-21T22:55:32.344 INFO:tasks.ceph:Archiving logs... 2024-01-21T22:55:32.344 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-01-19_10:13:34-fs-wip-vshankar-testing-20240119.075157-testing-default-smithi/7522024/remote/smithi138/log 2024-01-21T22:55:32.345 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-21T22:55:33.591 DEBUG:teuthology.misc:Transferring archived files from smithi152:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-01-19_10:13:34-fs-wip-vshankar-testing-20240119.075157-testing-default-smithi/7522024/remote/smithi152/log 2024-01-21T22:55:33.592 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /var/log/ceph -- . 2024-01-21T22:55:38.794 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_eb1b049f9003ac37e0a3ab33496ad1924d301254/qa/tasks/ceph.py", line 1935, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_eb1b049f9003ac37e0a3ab33496ad1924d301254/qa/tasks/ceph.py", line 252, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_eb1b049f9003ac37e0a3ab33496ad1924d301254/qa/tasks/ceph.py", line 346, in valgrind_post raise valgrind_exception Exception: valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun 2024-01-21T22:55:38.795 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-01-21T22:55:38.805 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_eb1b049f9003ac37e0a3ab33496ad1924d301254/qa/tasks/ceph.py", line 1935, in task mon0_remote.run( File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 54, in nested raise exc[1] File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_eb1b049f9003ac37e0a3ab33496ad1924d301254/qa/tasks/ceph.py", line 252, in ceph_log yield File "/home/teuthworker/src/git.ceph.com_teuthology_cd45576300487d997e5a85abed65500b9f5d143b/teuthology/contextutil.py", line 46, in nested if exit(*exc): File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_eb1b049f9003ac37e0a3ab33496ad1924d301254/qa/tasks/ceph.py", line 346, in valgrind_post raise valgrind_exception Exception: valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun 2024-01-21T22:55:38.805 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-01-21T22:55:38.805 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-21T22:55:38.813 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-01-21T22:55:38.956 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-01-21T22:55:38.968 INFO:teuthology.task.clock:Checking final clock skew... 2024-01-21T22:55:38.968 DEBUG:teuthology.orchestra.run.smithi138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-21T22:55:38.976 DEBUG:teuthology.orchestra.run.smithi152:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-01-21T22:55:38.992 INFO:teuthology.orchestra.run.smithi152.stderr:bash: line 1: ntpq: command not found 2024-01-21T22:55:38.998 INFO:teuthology.orchestra.run.smithi152.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-21T22:55:38.999 INFO:teuthology.orchestra.run.smithi152.stdout:=============================================================================== 2024-01-21T22:55:38.999 INFO:teuthology.orchestra.run.smithi152.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 39 +618us[ +618us] +/- 70ms 2024-01-21T22:55:38.999 INFO:teuthology.orchestra.run.smithi152.stdout:^* hv02.front.sepia.ceph.com 3 6 377 40 -549us[ -548us] +/- 52ms 2024-01-21T22:55:38.999 INFO:teuthology.orchestra.run.smithi152.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 104 +234us[ +236us] +/- 69ms 2024-01-21T22:55:38.999 INFO:teuthology.orchestra.run.smithi152.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-01-21T22:55:39.023 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-01-21T22:55:39.033 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-01-21T22:55:39.033 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-01-21T22:55:39.033 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 42 +607us[ +607us] +/- 70ms 2024-01-21T22:55:39.033 INFO:teuthology.orchestra.run.smithi138.stdout:^* hv02.front.sepia.ceph.com 3 6 377 46 -584us[ -575us] +/- 52ms 2024-01-21T22:55:39.033 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 +157us[ +157us] +/- 70ms 2024-01-21T22:55:39.033 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-01-21T22:55:39.035 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-01-21T22:55:39.045 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-01-21T22:55:39.045 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-01-21T22:55:39.061 DEBUG:teuthology.orchestra.run.smithi138:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-21T22:55:39.269 DEBUG:teuthology.orchestra.run.smithi152:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-01-21T22:55:39.341 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-21T22:55:39.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-21T22:55:39.443 DEBUG:teuthology.orchestra.run.smithi152:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-01-21T22:55:39.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-21T22:55:39.475 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-01-21T22:55:39.485 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-01-21T22:55:39.499 INFO:teuthology.task.internal:Duration was 2775.319826 seconds 2024-01-21T22:55:39.500 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-01-21T22:55:39.516 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-01-21T22:55:39.516 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-21T22:55:39.524 DEBUG:teuthology.orchestra.run.smithi152:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-01-21T22:55:39.569 INFO:teuthology.orchestra.run.smithi152.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-21T22:55:39.681 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-01-21T22:55:40.041 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-01-21T22:55:40.041 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi138.front.sepia.ceph.com 2024-01-21T22:55:40.042 DEBUG:teuthology.orchestra.run.smithi138:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-21T22:55:40.117 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi152.front.sepia.ceph.com 2024-01-21T22:55:40.118 DEBUG:teuthology.orchestra.run.smithi152:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-01-21T22:55:40.151 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-01-21T22:55:40.151 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-21T22:55:40.157 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-01-21T22:55:40.251 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-01-21T22:55:40.251 DEBUG:teuthology.orchestra.run.smithi138:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-21T22:55:40.257 DEBUG:teuthology.orchestra.run.smithi152:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-01-21T22:55:40.439 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-01-21T22:55:40.449 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-01-21T22:55:40.449 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-21T22:55:40.529 DEBUG:teuthology.orchestra.run.smithi152:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-01-21T22:55:40.558 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-01-21T22:55:40.571 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-21T22:55:40.579 DEBUG:teuthology.orchestra.run.smithi152:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-01-21T22:55:40.624 INFO:teuthology.orchestra.run.smithi152.stdout:kernel.core_pattern = core 2024-01-21T22:55:40.678 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = core 2024-01-21T22:55:40.727 DEBUG:teuthology.orchestra.run.smithi138:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-21T22:55:40.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-21T22:55:40.771 DEBUG:teuthology.orchestra.run.smithi152:> test -e /home/ubuntu/cephtest/archive/coredump 2024-01-21T22:55:40.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-01-21T22:55:40.786 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-01-21T22:55:40.796 INFO:teuthology.task.internal:Transferring archived files... 2024-01-21T22:55:40.797 DEBUG:teuthology.misc:Transferring archived files from smithi138:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-01-19_10:13:34-fs-wip-vshankar-testing-20240119.075157-testing-default-smithi/7522024/remote/smithi138 2024-01-21T22:55:40.797 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-21T22:55:40.908 DEBUG:teuthology.misc:Transferring archived files from smithi152:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-01-19_10:13:34-fs-wip-vshankar-testing-20240119.075157-testing-default-smithi/7522024/remote/smithi152 2024-01-21T22:55:40.908 DEBUG:teuthology.orchestra.run.smithi152:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-01-21T22:55:40.944 INFO:teuthology.task.internal:Removing archive directory... 2024-01-21T22:55:40.944 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-21T22:55:40.955 DEBUG:teuthology.orchestra.run.smithi152:> rm -rf -- /home/ubuntu/cephtest/archive 2024-01-21T22:55:41.003 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-01-21T22:55:41.016 INFO:teuthology.task.internal:Not uploading archives. 2024-01-21T22:55:41.016 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-01-21T22:55:41.032 INFO:teuthology.task.internal:Tidying up after the test... 2024-01-21T22:55:41.033 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-21T22:55:41.038 DEBUG:teuthology.orchestra.run.smithi152:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-01-21T22:55:41.053 INFO:teuthology.orchestra.run.smithi152.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 21 22:55 /home/ubuntu/cephtest 2024-01-21T22:55:41.085 INFO:teuthology.orchestra.run.smithi138.stdout: 393700 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jan 21 22:55 /home/ubuntu/cephtest 2024-01-21T22:55:41.088 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-01-21T22:55:41.101 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-01-21T22:55:41.121 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: Exception: valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun 2024-01-21T22:55:41.121 INFO:teuthology.nuke:Checking targets against current locks 2024-01-21T22:55:41.147 DEBUG:teuthology.nuke:shortname: smithi138 2024-01-21T22:55:41.147 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-21T22:55:41.170 DEBUG:teuthology.nuke:shortname: smithi152 2024-01-21T22:55:41.170 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-01-21T22:55:41.175 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-19_10:13:34-fs-wip-vshankar-testing-20240119.075157-testing-default-smithi/7522024', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-21 22:00:19.973383', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLyJGSaW2X8YY71fG8SngBVERcLbGclRlIadn2c0LipJRJCrI6qJ6FL8dD81NbSnvf5uuIDaFsJZ2adGvW7BFE4='} 2024-01-21T22:55:41.190 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi152.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-01-19_10:13:34-fs-wip-vshankar-testing-20240119.075157-testing-default-smithi/7522024', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-01-21 22:00:19.975340', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEeNVGhw4BYJUYb1i7A07lKn6tOaA9Y9fy0bgq+8084xBMhXWcHRfdSuRm+R4HCb2qW/u8uwFcdu2WAnl5J4ySI='} 2024-01-21T22:55:41.226 INFO:teuthology.orchestra.console.smithi138:Power off 2024-01-21T22:55:41.226 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-21T22:55:41.241 INFO:teuthology.orchestra.console.smithi152:Power off 2024-01-21T22:55:41.241 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-01-21T22:55:41.251 DEBUG:teuthology.orchestra.console.smithi138:power off output: Chassis Power Control: Down/Off 2024-01-21T22:55:41.251 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-21T22:55:41.267 DEBUG:teuthology.orchestra.console.smithi152:power off output: Chassis Power Control: Down/Off 2024-01-21T22:55:41.268 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-21T22:55:41.287 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2024-01-21T22:55:41.291 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is on 2024-01-21T22:55:45.288 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-21T22:55:45.296 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-21T22:55:45.398 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2024-01-21T22:55:45.406 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is on 2024-01-21T22:55:49.404 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-21T22:55:49.413 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-21T22:55:49.516 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is off 2024-01-21T22:55:49.523 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is on 2024-01-21T22:55:49.617 INFO:teuthology.orchestra.console.smithi138:Power off completed 2024-01-21T22:55:53.524 DEBUG:teuthology.orchestra.console.smithi152:pexpect command: ipmitool -H smithi152.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-01-21T22:55:53.635 DEBUG:teuthology.orchestra.console.smithi152:check power output: Chassis Power is off 2024-01-21T22:55:53.736 INFO:teuthology.orchestra.console.smithi152:Power off completed 2024-01-21T22:55:53.841 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 06652275eb934672217aaf9d4180f4a90395eb16 description: fs/verify/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{k-testing mount ms-die-on-skipped} objectstore-ec/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down mon-debug session_timeout} ranks/1 tasks/dbench validater/valgrind} duration: 2775.319826364517 failure_reason: 'valgrind error: Leak_StillReachable operator new[](unsigned long) UnknownInlinedFun UnknownInlinedFun' flavor: default owner: scheduled_vshankar@teuthology status: fail success: false 2024-01-21T22:55:53.841 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-01-21T22:55:53.918 INFO:teuthology.run:FAIL